-- Logs begin at Thu 2022-07-21 09:54:31 UTC, end at Thu 2022-07-21 11:07:58 UTC. -- Jul 21 10:26:09.048285 dv-cinder-849800-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 10:26:09.055543 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 10:26:09 2022] *** Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: nodename: dv-cinder-849800-2-iscsi Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: machine: x86_64 Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: clock source: unix Jul 21 10:26:09.057536 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: pcre jit disabled Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: detected number of CPU cores: 4 Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: current working directory: / Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: detected binary path: /usr/bin/uwsgi-core Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: your processes number limit is 47680 Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: your memory page size is 4096 bytes Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: detected max file descriptor number: 2048 Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: lock engine: pthread robust mutexes Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: thunder lock: enabled Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 10:26:09.057996 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 10:26:09.107771 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: Python main interpreter initialized at 0x555e66fe99c0 Jul 21 10:26:09.107771 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: python threads support enabled Jul 21 10:26:09.107771 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: your server socket listen backlog is limited to 100 connections Jul 21 10:26:09.107771 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: your mercy for graceful operations on workers is 90 seconds Jul 21 10:26:09.108168 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 10:26:09.108246 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: *** Operational MODE: preforking *** Jul 21 10:26:09.108508 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: *** uWSGI is running in multiple interpreter mode *** Jul 21 10:26:09.108508 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: spawned uWSGI master process (pid: 80584) Jul 21 10:26:09.108714 dv-cinder-849800-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 10:26:09.109830 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: spawned uWSGI worker 1 (pid: 80598, cores: 1) Jul 21 10:26:09.109830 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: spawned uWSGI worker 2 (pid: 80599, cores: 1) Jul 21 10:26:09.109830 dv-cinder-849800-2-iscsi devstack@keystone.service[80584]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 10:26:14.632388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 10:26:14.632388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes Jul 21 10:26:14.633171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 10:26:14.633171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes Jul 21 10:26:14.634289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 10:26:14.634289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: from cryptography.utils import int_from_bytes Jul 21 10:26:14.634786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 10:26:14.634786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: from cryptography.utils import int_from_bytes Jul 21 10:26:14.682441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 10:26:14.682649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 10:26:14.682851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] command line args: [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 10:26:14.683119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 10:26:14.683298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ================================================================================ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 10:26:14.683549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.683792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_dir = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.684516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.684698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_source = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.684809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 10:26:14.684924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] debug = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.685047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 10:26:14.685131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.686363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] command line args: [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 10:26:14.686431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.686504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 10:26:14.686601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.686670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ================================================================================ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 10:26:14.686730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.686799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] admin_token = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.686859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_dir = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_source = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_config_append = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] debug = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_dir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.687967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.688976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_config_append = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_dir = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.689933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.690901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] publish_errors = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.691814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] standard_threads = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.692931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] publish_errors = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_journal = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_json = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_stderr = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.693896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_syslog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] standard_threads = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.694558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.694649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.694877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.694995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.695072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.695190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_journal = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.695272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.695376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_json = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.695501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.695585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_stderr = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.695738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_syslog = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.695814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.695970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 10:26:14.696217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.696950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.697997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.698907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.699966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.700962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.701994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.702957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.703922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.704884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.705982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.706944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.707855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.708933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.709944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.710888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.711981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.712977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.713922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.714870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.715021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.715101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.715792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.715887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.716925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.717922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.718900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.external = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.719972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.password = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.token = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.720963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.721899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.external = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.722876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.password = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.723966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.token = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.724982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.725937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.726914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.727861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.728990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.729936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.730879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.731500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.731768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.731877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.732949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.733925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.734948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.735982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.736940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.737885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.738858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.739944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.740968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.741914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.742854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.743958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.744954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.745886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.746870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.747862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.748854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.749963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.750923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.751937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.752882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.753991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.754949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.755995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.756939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.757870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.758992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.759931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.760968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.761861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.762954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.763954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.764886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.765976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.766938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.767993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.768932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769330 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.769873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.770863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 10:26:14.771730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.771928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.772161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.772396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.772637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.772869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.773109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.773339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.773582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.773812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.774051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.774278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.774515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.774775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 10:26:14.774943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 10:26:14.803280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.803509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.803743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.805795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.806025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.806248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.812730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.815602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.841343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.841563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.842872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.843103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.986745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:14.988075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.037862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.038057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.038163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.038248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.104912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.105950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.106217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.106297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.106496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.132601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.132879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.133879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.134004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.145917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.146222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.159443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.159673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.160040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.162505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.162689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.162801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.162801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.163223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 10:26:15.178044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.178252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 10:26:15.178611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.178815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 10:26:15.179013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.179098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.179223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 10:26:15.179334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.179556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.179640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.179756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 10:26:15.179837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.180022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.180362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.180620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.180872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.191988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.192264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.192527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.192807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.193207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.193495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.193761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.194039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.201599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.204905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.205184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.205414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.205498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.205852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.206134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.206393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.206691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.206965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.207233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.207505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.207771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.211860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.212349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.212839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.213517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.214020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.214483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.214995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.215523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.215965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.216428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.216822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.234955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.236598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.236922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.242178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.242458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.242763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.247557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.249468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.249746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.250857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.251142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.255104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.255371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.255645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.255799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.256086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.256367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.256648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.256922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.257198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.257469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.257738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.265973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.266460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.273322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.273604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.273885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.274163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.274433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.274732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.275000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.275264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.277589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.277963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.282882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.286488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.286808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.292148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.295561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.295836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.295942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.299781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.300124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.300411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.300694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.300980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.301258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.303446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.310995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.311500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.319111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.322500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.322873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.323157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.323622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.323996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.329318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.329619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.329988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.330277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.330566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.330861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.334098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.336163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.336477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.339630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.339911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.340217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.340558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.340946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.358457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.358795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.359089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.359374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.359655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.359998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.360356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.368803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.369232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.369533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.369890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.384803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.388393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.388745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.390244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.390514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.392092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.393834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.393958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.394205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.394681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.395014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.395309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.395946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.396230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.396516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.404159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.405777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.409121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.409230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.409409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.409498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.409692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.409777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.409972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.410054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.410255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.410566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.410850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.419750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.420036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.420316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.428201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.428480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.428759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.429037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.429306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.429583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.429861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.430131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.430405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.430556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.430680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.430955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.431213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.431471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.431740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.434371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.437699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.439344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.440989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.441271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.441558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.449028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.451378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.454874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.455165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.455449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.455724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.464993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.465336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.465623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.471208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555e66fe99c0 pid: 80599 (default app) Jul 21 10:26:15.473766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 10:26:15.474053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.474335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.474644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.474736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e220cd-d160-4c1e-a98b-264e2b52189a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:15.474916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.475036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e220cd-d160-4c1e-a98b-264e2b52189a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:15.475197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.475283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e220cd-d160-4c1e-a98b-264e2b52189a None None] PATH_INFO: `/v3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:15.475481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.475750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.476023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.476293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.476576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.476838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.477101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.477369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 10:26:15.479446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Thu Jul 21 10:26:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:15.517440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555e66fe99c0 pid: 80598 (default app) Jul 21 10:26:24.314154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:24.314760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:24.314760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:24.316051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 2/2] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:24.324024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:24.324677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:24.324677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:24.395020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: INFO dbcounter [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] Registered counter for database keystone Jul 21 10:26:24.417302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG oslo_db.sqlalchemy.engines [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80598) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 10:26:24.424378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writer thread running {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 10:26:24.608566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG passlib.handlers.bcrypt [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80598) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 10:26:24.608981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG passlib.handlers.bcrypt [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80598) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 10:26:24.664988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:24.666639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:24.795807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1/3] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 10:26:24.967522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:25.038474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: INFO dbcounter [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] Registered counter for database keystone Jul 21 10:26:25.058389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG oslo_db.sqlalchemy.engines [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80599) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 10:26:25.065330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writer thread running {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 10:26:25.184963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd8b36-5df0-4937-83b1-eed336ae0787 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:25.189292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:25.189954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:25.190242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:25.190518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:25.199321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:25.253798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 10:26:25.254143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37d5ec-bc65-4250-9143-d941611b0656 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:25.265123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 3/4] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Jul 21 10:26:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:26:25.271220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:25.288903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f15a3-7853-405b-8328-cefe7a084553 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:25.294293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:25.295457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:25.295742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:25.296020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] PATH_INFO: `/v3/projects/549a3510fe3f48278464436ac5f2a88c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:25.297537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] RBAC: Authorizing `identity:get_project(project_id=549a3510fe3f48278464436ac5f2a88c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:25.350067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 10:26:25.350716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859f78-575e-497d-bfa3-bcb00da2f753 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:25.355158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 2/5] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Jul 21 10:26:25 2022] GET /identity/v3/projects/549a3510fe3f48278464436ac5f2a88c => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:27.392904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadfa3b-1362-42b3-98d6-485e7ec185bb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:27.393455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadfa3b-1362-42b3-98d6-485e7ec185bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:27.393455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadfa3b-1362-42b3-98d6-485e7ec185bb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:27.397580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 4/6] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:27.403652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:27.404125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:27.404371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:27.432231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:27.434298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:27.442769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 3/7] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 10:26:27.466762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadfa3b-1362-42b3-98d6-485e7ec185bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:27.479099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadfa3b-1362-42b3-98d6-485e7ec185bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:27.483970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:27.484609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:27.484890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:27.485163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:27.493465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:27.495265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf4e20-d4e4-4aca-bb1c-882f0fee2141 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:27.505569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 5/8] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Jul 21 10:26:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:26:27.511359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:27.521328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dffb6-679a-4e31-98c1-ea3db48e18ed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:27.525745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:27.526650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:27.526935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:27.527211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:27.529121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:27.530999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a13a71-c552-4a16-86e2-5bbf3f64bf4b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:27.532601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 4/9] 10.222.0.48 () {56 vars in 1198 bytes} [Thu Jul 21 10:26:27 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:29.676209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0542afb-ba77-42b5-bb57-2707fd979f2b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:29.676733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0542afb-ba77-42b5-bb57-2707fd979f2b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:29.676733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0542afb-ba77-42b5-bb57-2707fd979f2b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:29.678120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 6/10] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:29.684382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:29.684846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:29.685015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:29.702663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:29.703857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:29.728462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab752-56a3-4e21-a7ca-a7967e9e61fe None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:29.728747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab752-56a3-4e21-a7ca-a7967e9e61fe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:29.729025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab752-56a3-4e21-a7ca-a7967e9e61fe None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:29.730369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 7/11] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:29.736975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:29.737262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:29.737544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:29.781348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80599) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 10:26:29.781769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80599) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 10:26:29.799851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 5/12] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:29.824343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:29.833365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:29.833365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:29.852251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:29.956960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebec0fd-df2d-48d2-805a-82c5e028b275 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:29.963071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 8/13] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:26:29.997054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.003768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.004577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.004856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.005128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.020297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:30.039369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:30.040398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:30.041229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 6/14] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:26:30.048659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.126389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501c2f0-bd54-4673-ac82-3b6d96d7327c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.128978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.139551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.168860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23106-ea4d-4dd9-b1de-07c622464177 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.171493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.171493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.171493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.171493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.209217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.210932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.210932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.210932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.210932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:30.210932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.213248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.232535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 7/15] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:30.240873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.253873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.255513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.360139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa834-46cf-4c30-aa35-ab274010db90 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.364918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 9/16] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:26:30.371960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.375573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.399528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.400415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.400704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.400982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.402216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.405177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.407158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 8/17] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:30.413186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.430617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.491497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a38a1-1d4e-4d6c-92d4-3f5f52a637c4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.527835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84021fff-3bbe-41a7-8291-c5508227b01e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.528226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.529040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.529323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.529592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.549496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.554943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.570080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.570902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.571185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.571462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.572672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:26:30.573393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:30.575380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:30.577329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:26:30.577329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:26:30.577329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:26:30.577329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:30.577670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.577841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:26:30.578053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:30.581857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.584582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:30.644104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.673240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:30.692394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e8d3-36dd-4d05-8fcb-32e54937c8e5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.693114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69d74f-5410-4b2d-9883-f5840f0a06c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:30.727167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.727588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:30.727836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.728117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.728254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:30.728469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.728565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:30.728797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:30.729235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:30.729555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.729641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.domain.id=default, user.id=1db1bf9bea374f37b124738f1edc4d39) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:30.729934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:30.731952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0a5d-6b14-443b-b2df-4b73afd927fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.732154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2ef83-8d6d-4449-b328-5e8dfce8c1c4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:30.739799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/role_assignments?user.id=1db1bf9bea374f37b124738f1edc4d39&scope.domain.id=default&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:30.752142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:33.415734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b6c8b-b21b-4e07-b640-2cf6df6040a4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.416736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b6c8b-b21b-4e07-b640-2cf6df6040a4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.417259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b6c8b-b21b-4e07-b640-2cf6df6040a4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.419636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:33.428250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0c30-86c7-4a0d-9d10-d40216843edc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.429183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0c30-86c7-4a0d-9d10-d40216843edc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.429661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0c30-86c7-4a0d-9d10-d40216843edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.464748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-302f0c30-86c7-4a0d-9d10-d40216843edc None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:33.467368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-302f0c30-86c7-4a0d-9d10-d40216843edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:33.568448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:26:33.617612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b6c8b-b21b-4e07-b640-2cf6df6040a4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:33.757027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c91e-6edb-42b6-963f-53c0c9b73aef None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.757641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c91e-6edb-42b6-963f-53c0c9b73aef None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.758111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c91e-6edb-42b6-963f-53c0c9b73aef None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.759692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 13/24] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:33.765895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.766582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.767070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.781012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b6c8b-b21b-4e07-b640-2cf6df6040a4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:33.805422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:33.806992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:33.879021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:33.880230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.880704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.881169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.897883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:33.899875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:33.909819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:33.910847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-07a9e219-3f91-4902-adb5-0572f23e46eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:26:33.941987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:33.942896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:33.944079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:26:33.944079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:26:33.944079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:26:33.944079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:33.946151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 12/25] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:33.947723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b08a-3170-4339-befa-54269389a8aa None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.948222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b08a-3170-4339-befa-54269389a8aa None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.948686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b08a-3170-4339-befa-54269389a8aa None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.951407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 13/26] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:33.957927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:33.958523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:33.959037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:33.981136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 14/27] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 10:26:33.982151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:33.985147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:33.986976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:34.110307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 14/28] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:34.111493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:34.158214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229919-e7ec-48fc-ba04-2fc4261f597c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:34.227568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:34.228251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.228533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.228815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.229805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:34.230123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:34.233033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033575f5-60c7-49f4-a8de-8beb6eacb1cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:34.249281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 15/29] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:34.250575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510157-da2e-45d4-a6bf-181716361e11 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.250854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510157-da2e-45d4-a6bf-181716361e11 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.251134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510157-da2e-45d4-a6bf-181716361e11 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.252208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf062fc6-e90a-4c0c-99bb-5a179fd7ddaa None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:34.252431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 16/30] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:34.253132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af510157-da2e-45d4-a6bf-181716361e11 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:34.291920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:34.302965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:34.303602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:34.304629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.305066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.305642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.318134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:34.327998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:34.328916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:34.329650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:34.332178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 15/31] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:34.333243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:34.389834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af510157-da2e-45d4-a6bf-181716361e11 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:34.427324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:34.428011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.428298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.428853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.429369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:34.430759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477bc99-fee1-40cf-b9fc-9365d0b6389e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:34.455932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0581cd-07c6-4ae0-b56f-a30bd01747d1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:34.499314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:34.500359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.500789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.501208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.502081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:34.503740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:34.632658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 17/32] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 21 10:26:34 2022] POST /identity/v3/roles => generated 225 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:34.635869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:34.636585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:34.637817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:34.672435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:34.675594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:34.758571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 16/33] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:26:34 2022] POST /identity/v3/projects => generated 301 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:34.758571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:34.814708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:34.819966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 18/34] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:34.857083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:34.894395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98d7a0-e51c-4dc7-b6fb-9043f885c456 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:34.927756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:35.000467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.001190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.001479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.001767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.002590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:35.002902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.004531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.022314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 17/35] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:35.033033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.039507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe3c2b-a2ea-4db8-a395-d9cb1e8dc5a8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.094333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.098569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.098569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.098569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.098569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.099882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf32ed-955a-4289-aa45-5b7d5a762296 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.193633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 19/36] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 21 10:26:34 2022] POST /identity/v3/roles => generated 219 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:35.195078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059524-e016-4931-8f43-86c0c56ef09f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.195466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059524-e016-4931-8f43-86c0c56ef09f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.195863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5059524-e016-4931-8f43-86c0c56ef09f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.197237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 20/37] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:35.198499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9a3c9-1c70-4258-a1c5-fe6786743a24 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.198815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9a3c9-1c70-4258-a1c5-fe6786743a24 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.199090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9a3c9-1c70-4258-a1c5-fe6786743a24 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.201301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 21/38] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:35.206812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.207168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.207449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.215121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:35.219111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa744e5-2f0f-4f3f-927a-be7b91d5ac4d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.227041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:35.228282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:35.266269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.267219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.267503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.269239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.269950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.271614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e991f-af20-4f9e-8d20-a290ac7fd5ad None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.273672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 18/39] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:35 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:35.274912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.275199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.275535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.295933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:35.297579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:35.317176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 22/40] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:35.318347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.350795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:35.398230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 19/41] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:35.399307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.472836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871aa6-e04c-4b3e-b1cf-380330573c2f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.516202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.517029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.517309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.517583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.519222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.520735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.522808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 23/42] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:35.523433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.618474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:35.620934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:35.625400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae51994-d86a-42cb-b30e-a60a02516ddf None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.636699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998dd00-bf1c-4d3e-85c6-6ca1cb94cb68 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.662071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.662789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.663067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.663342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.664095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.665480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.679404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.680201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.680484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.680762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.681923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.683403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.686166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 24/43] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:35.686926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.782241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 20/44] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 21 10:26:35 2022] POST /identity/v3/roles => generated 223 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:35.783389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:35.798420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543859-12c0-4376-9a3b-3d0dc4761d98 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.843213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.844178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.844467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.844749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] PATH_INFO: `/v3/domains/default/users/1db1bf9bea374f37b124738f1edc4d39/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.847359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1db1bf9bea374f37b124738f1edc4d39, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.848943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dc872-43a7-49a2-a116-1ff6412ee875 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:35.907640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe209-f399-4bbb-8511-60f39a691d53 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:35.919813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 25/45] 10.222.0.48 () {58 vars in 1361 bytes} [Thu Jul 21 10:26:35 2022] PUT /identity/v3/domains/default/users/1db1bf9bea374f37b124738f1edc4d39/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:35.964096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:35.965436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:35.965735 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:35.966015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:35.966800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:35.968298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb07b1-979e-46d0-906e-9f7bd4d5336d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:36.051501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 21/46] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:26:35 2022] POST /identity/v3/projects => generated 312 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:38.016645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07f06-0af4-4bc1-9b4f-b477ae31179d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.017168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07f06-0af4-4bc1-9b4f-b477ae31179d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.017277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07f06-0af4-4bc1-9b4f-b477ae31179d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.018573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 26/47] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:38.023854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d291-5721-4689-bf68-e9cc858ddda2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.024257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d291-5721-4689-bf68-e9cc858ddda2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.024424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d291-5721-4689-bf68-e9cc858ddda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.041923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0e22d291-5721-4689-bf68-e9cc858ddda2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:38.043894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0e22d291-5721-4689-bf68-e9cc858ddda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:38.046872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.111948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 22/48] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:38.135093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa07f06-0af4-4bc1-9b4f-b477ae31179d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.143821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28abb5-b746-4df9-ba22-9ea781d32fe6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.144115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28abb5-b746-4df9-ba22-9ea781d32fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.144394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28abb5-b746-4df9-ba22-9ea781d32fe6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.145760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 23/49] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:38.150731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210d20-476f-4049-9767-a815833d6034 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.151185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210d20-476f-4049-9767-a815833d6034 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.151301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.151465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210d20-476f-4049-9767-a815833d6034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.169423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a3210d20-476f-4049-9767-a815833d6034 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:38.170651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a3210d20-476f-4049-9767-a815833d6034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:38.240181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 24/50] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:26:38.245170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa07f06-0af4-4bc1-9b4f-b477ae31179d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.264699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3210d20-476f-4049-9767-a815833d6034 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.342168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.376866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3210d20-476f-4049-9767-a815833d6034 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.385780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.387192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.387549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.387837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.399407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:38.409339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:38.410269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:38.411059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 27/51] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:38.411732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.412528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.412810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.413083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.414362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:38.415855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:38.421921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 25/52] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:38.422063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.423820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.490881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.521238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c0a3b-b200-422e-8575-07bfb22aa4dc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.527274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c670ab6-5b7d-4264-af98-ccd1e725cc89 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.555887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.556567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.556851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.557134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.557885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:38.558194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:38.560386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:38.561758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.562454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.562790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.563164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.564141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:38.565296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:38.572681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 28/53] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:38.575034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.578941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.679595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526ddcc-947c-4fe3-a037-1d3486fdcb5c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.715281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.716168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.716434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.716707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.717896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:38.719660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:38.720978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 29/54] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:38.726327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.729062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:38.765780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 26/55] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Jul 21 10:26:38 2022] POST /identity/v3/projects => generated 298 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:38.834778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bd6a-643e-42d0-afc3-b0ada09f6d47 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:38.873738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:38.874576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:38.874865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:38.875143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:38.876391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:38.877870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1eda7-fa27-46a8-872a-3e6c6d124f44 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:38.879586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 30/56] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:38.885649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:38.999859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d79a-5b40-40b4-8442-6a659e2a5f7f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:39.036024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:39.036697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:39.037128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:39.037324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:39.038116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.domain.id=default, user.id=1db1bf9bea374f37b124738f1edc4d39) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:39.038476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:39.039922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc6beb-76b3-496a-b559-5bdf8d57710f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:39.046438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 27/57] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Jul 21 10:26:38 2022] GET /identity/v3/role_assignments?user.id=1db1bf9bea374f37b124738f1edc4d39&scope.domain.id=default&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:40.836904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a413-3018-4ff7-872a-bdbf116eacab None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:40.837426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a413-3018-4ff7-872a-bdbf116eacab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:40.837513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a413-3018-4ff7-872a-bdbf116eacab None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:40.838842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 31/58] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:40.844643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:40.845069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:40.845234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:40.862087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:40.863362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:40.915615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:40.932563 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 28/59] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:40.960870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a413-3018-4ff7-872a-bdbf116eacab None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:41.033099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:41.065510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a413-3018-4ff7-872a-bdbf116eacab None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:41.103290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:41.104003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:41.104292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:41.104575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:41.106318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:41.107819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a08332-520c-4d05-bad6-8d62ea1686c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:41.109460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 32/60] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:41.115118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:41.218191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c1d25-5f5e-4cf2-98c8-17f859de18e2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:41.253095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:41.253816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:41.254096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:41.254377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:41.255121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:41.256565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed0b13-7f93-4ff0-ae43-175e7fd498db None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:41.341828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 29/61] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:26:41 2022] POST /identity/v3/users => generated 275 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:44.767800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91ac4-f9f5-4dbe-8f43-077abeb50956 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:44.768336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91ac4-f9f5-4dbe-8f43-077abeb50956 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:44.768336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec91ac4-f9f5-4dbe-8f43-077abeb50956 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:44.769724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 33/62] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:44.775214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:44.775618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:44.775784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:44.802915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:44.805321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd124-cadb-4813-a1d3-d9050bb351fb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:44.805604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd124-cadb-4813-a1d3-d9050bb351fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:44.805876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd124-cadb-4813-a1d3-d9050bb351fb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:44.807488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 34/63] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:44.808210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:44.815896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d4770-978d-4bec-8fc7-31fae955a752 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:44.816486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d4770-978d-4bec-8fc7-31fae955a752 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:44.816957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d4770-978d-4bec-8fc7-31fae955a752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:44.848546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:44.849862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2b6d4770-978d-4bec-8fc7-31fae955a752 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:44.851289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b6d4770-978d-4bec-8fc7-31fae955a752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:44.885064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 30/64] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:44.910885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:44.998157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 35/65] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:45.002648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059033-147d-4724-abab-c168192df12a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.002648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059033-147d-4724-abab-c168192df12a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.002648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059033-147d-4724-abab-c168192df12a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.014583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 36/66] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:45.015630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.015934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.016245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.068184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:45.074926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:45.099037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a0048-ea12-4fa9-a8ac-6b8cf95776b1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.142950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.144539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.145331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.146001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.159998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:45.166793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:45.169760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:45.170675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:45.171637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:45.174496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 31/67] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:45.180474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.182469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 37/68] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:45.188946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.248970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.368273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a49f-a782-4674-8e2d-68eff4f38dcc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.378922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e8845-2a80-4305-95e5-afd6530f181b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.415313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.415992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.416284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.416567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.417330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:45.417641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:45.419227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:45.432268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.433198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 38/69] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:45.434639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.435234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.435611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.436271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.450738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6f05e73a-855a-40b5-85c3-263c230badfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:45.458784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:45.459703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:45.460494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 32/70] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:45.461674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2a83-c08a-4c04-bb91-d44b58b514b3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.461945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2a83-c08a-4c04-bb91-d44b58b514b3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.462226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2a83-c08a-4c04-bb91-d44b58b514b3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.463876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 33/71] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:45.464504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b2a83-c08a-4c04-bb91-d44b58b514b3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.504918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.555153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28916-5bd4-447c-a069-d6f21cae870b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.564767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.576170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b2a83-c08a-4c04-bb91-d44b58b514b3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.594099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.594941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.595224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.595496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.605830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:45.612865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.613653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.613939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5fe33ef5-6b61-487c-86a9-1c18b0316194 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:45.614201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:45.615176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:45.615977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:45.616550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.617351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 39/72] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:45.617351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebde7f-c6f2-46e7-97c7-e0a5c9d5dfcf None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.617351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebde7f-c6f2-46e7-97c7-e0a5c9d5dfcf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.617351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebde7f-c6f2-46e7-97c7-e0a5c9d5dfcf None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.622917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 40/73] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:45.622917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebde7f-c6f2-46e7-97c7-e0a5c9d5dfcf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.632543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:45.634111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594b6e7-43b2-4c3e-9749-64023ba6834b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:45.635879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 34/74] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:45.638447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.638764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.639042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.656678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:45.657946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:45.735117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 35/75] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:45.736798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.757177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebde7f-c6f2-46e7-97c7-e0a5c9d5dfcf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.781501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.796819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.797556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.797845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.798121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.798906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:45.799215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:45.800649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545b533-57cf-408f-97d5-1b2f25471c1d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:45.809819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 41/76] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:45.811377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.811671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.811950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.833156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:45.834506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:45.844664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:45.866164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2681e4-dd24-4648-9f66-4a9ad32f3096 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:45.905855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:45.906571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:45.906850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:45.907125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:45.907891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:45.908204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:45.908287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 42/77] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:45.909142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:45.909694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:45.923445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 36/78] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:45.923445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.023944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7841-a1c1-49fe-a601-20001feacb46 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.031918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143f130-74a2-4655-a3ec-89983d93ee74 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.040191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.067579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.067823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.068268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.068548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.068634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.068862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.069010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.069137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.079717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.083650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:46.091582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:46.095610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 37/79] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:46.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.158566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.160345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.238352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd1218-d1ea-4a70-870f-e4840a21a435 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.278052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.278964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.279250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.279528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.280752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.282279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.284159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:46.284851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.290403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 43/81] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:46.290403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.304937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.361772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.396358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a16f6-956d-43b2-accb-3259af94092d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.396757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ff9fc-57f7-4068-9ecc-b9823de60231 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.431893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.432085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.432606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.432716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.432892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.432999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.433181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.433308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.435217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.435320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.436677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.436773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.438482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 39/82] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:46.438607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 44/83] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:46.440292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.443689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.534341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.554419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207fa9-2bd1-450c-b94a-9761611e8ac9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.554968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f91663-3f76-4b91-b426-f93cae5a7c0f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.591566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.592250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.592537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.592818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.593585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:46.593904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.594731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.595415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.595688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.595787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.596001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.598422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.606290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 40/84] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:46.607017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.668751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.670233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.717045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85d40-3c8b-4a21-a657-b6f452d297f8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.752525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.753875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.754765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.755048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.755323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.756504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.757965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.759810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 41/85] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:46.760583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.773758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 45/86] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:46.774630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:46.829847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:46.868269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ffb-041c-49b1-8e99-7e5885533904 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.878862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45179bf8-e23f-4d15-8ac4-9ead596bf9dd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:46.904560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.905215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.905493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.905776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.916210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:46.916884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:46.917170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:46.917449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:46.918169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.919665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:46.974758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:46.976159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.004451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 46/87] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:26:46 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:47.005325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.054782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.065177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.097286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 42/88] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:47.098289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.144410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ac863-3d46-4bb9-b841-7b003584c924 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.190504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.191855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.194447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.194872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.197710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.198621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.200570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 47/89] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:47.201375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.243944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598b717-2362-4832-b67b-2715ec9ccc0b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.297309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.297986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.298270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.299271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.300136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:47.300749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.302610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.311470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 43/90] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=a0903d733c974f51970116b3948f5ff8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:47.312484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.361355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.361789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c874a9-7c9c-4960-a120-325997a29d73 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.366897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.426641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.427569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.427863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.428164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.429576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.431393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.442178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 48/91] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:47.443276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.456174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bee0-d315-4050-b9e5-80afb5863a26 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.499265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.500070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.500352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.500630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.503130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.503562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.506162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 44/92] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:47.506924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.693742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.699036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece1662-a8c0-412d-8378-870f1fb2015d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.702324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9f439-521d-4a80-8f71-494c51c24eba None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.768286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.769301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.769787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.770265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.771451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=1db1bf9bea374f37b124738f1edc4d39) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:47.772349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.775378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.787293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.787828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 45/93] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/role_assignments?user.id=1db1bf9bea374f37b124738f1edc4d39&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:47.787989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.788260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.788566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.790860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:47.882151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:47.921178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45513-3661-40f7-ab17-b2f184183f59 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:47.947681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.950486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.981411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:47.982341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:47.982824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:47.983370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:47.984363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:47.984796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:47.986412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:47.997831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 46/94] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:48.141803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 49/95] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:48.147679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:48.190408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:48.302221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51503f2-5f91-467c-aea5-d9d3a3e7df4c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:48.362490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:48.364021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:48.364485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:48.365005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] PATH_INFO: `/v3/projects/c619f6c91b6345d8a519f24f4b3e3368` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:48.366753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] RBAC: Authorizing `identity:get_project(project_id=c619f6c91b6345d8a519f24f4b3e3368)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:48.368317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccbaf2-0059-4f51-a363-82acb705ba55 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:48.370314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 47/96] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:48 2022] GET /identity/v3/projects/c619f6c91b6345d8a519f24f4b3e3368 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:48.378706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:48.487542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:48.531644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951453-af58-46fe-a8c4-4601a945b8c5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:48.581107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:48.581814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:48.582102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:48.582379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:48.583283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=c619f6c91b6345d8a519f24f4b3e3368, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:48.583600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:48.585082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933acc-d17c-40df-83fd-d9533012b7a9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:48.602187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 50/97] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:48 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=c619f6c91b6345d8a519f24f4b3e3368&role.id=a0903d733c974f51970116b3948f5ff8 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:50.592598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5882-2307-46a3-abb3-f7bb2f3e278a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.593259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5882-2307-46a3-abb3-f7bb2f3e278a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.593259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5882-2307-46a3-abb3-f7bb2f3e278a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.594449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 48/98] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:50.600455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0177a-df69-407d-a9ee-284ebfb7119a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.601313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0177a-df69-407d-a9ee-284ebfb7119a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.601784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0177a-df69-407d-a9ee-284ebfb7119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.624015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-85d0177a-df69-407d-a9ee-284ebfb7119a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:50.625550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-85d0177a-df69-407d-a9ee-284ebfb7119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:50.662898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df38f-79a1-4a43-8c3f-0050b3886a1b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.663636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df38f-79a1-4a43-8c3f-0050b3886a1b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.664186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df38f-79a1-4a43-8c3f-0050b3886a1b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.665847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 49/99] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:50.671785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529cbb6-19fa-4ded-a55a-4c364f5b85e4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.672485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529cbb6-19fa-4ded-a55a-4c364f5b85e4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.673113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529cbb6-19fa-4ded-a55a-4c364f5b85e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.700814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e529cbb6-19fa-4ded-a55a-4c364f5b85e4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:50.702396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e529cbb6-19fa-4ded-a55a-4c364f5b85e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:50.728111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 51/100] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:50.760768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1f82-d47d-4c77-b9b8-71d10aa08000 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.761078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1f82-d47d-4c77-b9b8-71d10aa08000 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.761346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1f82-d47d-4c77-b9b8-71d10aa08000 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.764106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 52/101] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:50.765746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27ee8-90b2-45a3-96a8-97bee4d6095f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.766293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27ee8-90b2-45a3-96a8-97bee4d6095f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.766813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27ee8-90b2-45a3-96a8-97bee4d6095f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.768648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 53/102] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:50.769967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.770458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.771232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.796260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:50.797958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:50.828255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 50/103] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:26:50.830965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:50.831719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:50.832456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:50.852153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:50.853680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:50.880273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:50.893167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:50.903109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 54/104] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:50.903983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:50.956460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 51/105] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:50.957483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.024817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d78837-ad2e-4d1a-abbd-5c64e3e3c5fb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.073545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.075234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.075515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.075790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.087594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cb57cbd7-7ae7-42bc-8680-389c3053ea60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:51.096401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:51.097598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:51.099071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 55/106] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:51.100193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42256-1eac-416f-9f23-3c4c76d2ea11 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.100193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42256-1eac-416f-9f23-3c4c76d2ea11 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.100193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42256-1eac-416f-9f23-3c4c76d2ea11 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.106409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 56/107] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:51.106409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb42256-1eac-416f-9f23-3c4c76d2ea11 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.109938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085d8d4-4daa-4bcc-82a9-7193e99c28b1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.153494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.154388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.154703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.154980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.173060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:51.185809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:51.190909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:51.191839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 52/108] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:51.192695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.219138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.272593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb42256-1eac-416f-9f23-3c4c76d2ea11 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.315929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.316411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.316696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.316973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.318161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:51.319804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:51.321596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 57/109] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:51.322209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.343426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e19d4e-0e68-4eea-a850-2bbfdecf4baf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.401384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.402432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.402797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.403067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.416810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2ec6ec01-3980-4f09-b944-c73b5bfc4212 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:51.425579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:51.426597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 53/110] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:51.432553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.433907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.433907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.447114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:51.448404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:51.470674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e161c-c93d-41f7-b432-23b260c61e30 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.497521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.513770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.514488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.514791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.515055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.516034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:51.516212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:51.517537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:51.524899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 54/111] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:51.532152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 58/112] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:51.532152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.533053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.536839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.647912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b231f05-2f82-4d11-ba43-4b01fba03bf6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.661435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463af12-7029-431e-9bb7-083147c281e9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.685336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.686193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.686332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.686582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.687291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:51.688776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:51.698564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.699366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.699655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.700336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.701114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:51.701424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:51.702858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:51.711556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 55/113] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:51.712438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.776757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 59/114] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:26:51 2022] POST /identity/v3/projects => generated 302 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:26:51.777937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.783565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.827236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c280c-380b-414f-ae37-c38d8e8282ff None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.836230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:51.875241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.876095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.876484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.876868 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:51.877752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:51.878174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:51.879722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:51.891990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 56/115] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:51.892989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:51.930983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a189cbb-5015-45d5-88dd-02dd445233a8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:51.979776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:51.982776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:51.982776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:51.982776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] PATH_INFO: `/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.006488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] RBAC: Authorizing `identity:get_user(user_id=f34b5a5061d84b4e92e9cf33bf93cb21)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.009071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559081-c9bb-4356-b614-183c9af80267 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.012362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 60/116] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.014300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cafe8-9858-4b81-b551-dd2410498213 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.014903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cafe8-9858-4b81-b551-dd2410498213 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.015410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cafe8-9858-4b81-b551-dd2410498213 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.017695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 61/117] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:52.018964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cafe8-9858-4b81-b551-dd2410498213 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.036581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5a4a-e863-453a-bae3-75c298fe4de7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.070199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.081622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.085693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.085693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.085693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.095202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:52.103492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:52.104387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:52.105198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 57/118] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:52.106517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.173152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.180034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cafe8-9858-4b81-b551-dd2410498213 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.222113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.223002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.223284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.223560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.224752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.226267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.233421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 62/119] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.233421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.272708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ffff-8b09-4f99-9138-3a6ece707ea6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.313379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.314742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.315249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.315832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.318444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.320369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6fb70-8b48-45f3-8ec2-cc608ba779f7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.322258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 58/120] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.323759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.324605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.325039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.343166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:52.344670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:52.351583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9219c0-74b1-47ea-8225-c638de5e741e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.394683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.395346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.395639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.395914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.405291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.425051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 59/121] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:52.425910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.442252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.494143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.495579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.555225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e8502-be0b-40a7-b96c-32d7e81ebff9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.605050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.605955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.606368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.606804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.607715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:52.608139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.610060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.622418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 63/122] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:52.627372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 60/123] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.627834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.628970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.701196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.725031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.762797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f23004-fa15-499f-90d8-1a6e04ce9ea1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.773696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61145f79-11da-4b38-aee0-b1b0ae798466 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.809594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.810620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.810970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.811315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.812699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.814247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.816190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 61/124] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.817202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.828043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.829000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.829342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.829681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:52.830996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:52.832470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:52.834297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 64/125] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:52.835106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:52.946120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b538-cf78-4eed-a243-21e9fd16a0bd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.965429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:52.966873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765c7b1-a510-462c-bf3e-7c770a3d7bd3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:52.988146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:52.989005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:52.989294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:52.989582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.003536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:53.008565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:53.012258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:53.013241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:53.014008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 62/126] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:53.014691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.019508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.020392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.020673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.036776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.036776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.036776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.036776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 65/127] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:52 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:53.036776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.164794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d06ef-175a-4a02-8d99-46244e71e777 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.172129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355095d1-15ac-4304-b3a7-306f0c010ffd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.215602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.216797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.217148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.217517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/ed5bda805a18497ab72e318c7b4698e5/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.218263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.219192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.219517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.219745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.220334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, user_id=ed5bda805a18497ab72e318c7b4698e5, role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.220944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.222031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.222440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.229225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 63/128] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:53 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:53.229225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.293646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:53.313856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 66/129] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:26:53 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/ed5bda805a18497ab72e318c7b4698e5/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:53.314806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.369576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7c017-ebaa-411e-a4ba-706c97369e18 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.423044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:53.424529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.425638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.426007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.426360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/ed5bda805a18497ab72e318c7b4698e5/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.428776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, user_id=ed5bda805a18497ab72e318c7b4698e5, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.430385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.498656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c03022-7a70-44da-a789-d14c708df73e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.544049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.544665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.544952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.545223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.546050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:53.546419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.547914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.552301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 64/130] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:26:53 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/ed5bda805a18497ab72e318c7b4698e5/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:53.556479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.563004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 67/131] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:53.563998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.769301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436404b-f299-4440-8caf-c2370ce99b3b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.772032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:53.799280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1d9c0-a8df-49a9-bf6d-381ee31288d9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:53.823458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.824766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.825260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.825951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.826890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=f34b5a5061d84b4e92e9cf33bf93cb21) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:53.827209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.829178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.838232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 65/132] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:53 2022] GET /identity/v3/role_assignments?user.id=f34b5a5061d84b4e92e9cf33bf93cb21&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:53.839188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:53.839389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.848155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:53.849052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:53.853897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:53.853897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:53.853897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:53.853897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:53.854247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 68/133] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:53 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:53.871803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:53.990708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446761d1-9c76-4ecb-8791-8a44557009ce None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.034118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.035060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.035358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.035702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.036970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.038465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:54.040246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 66/134] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:53 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:54.051754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:54.056325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac0c07-2b60-4c63-9180-680314603c86 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.104591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:54.132432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.133366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.133651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.133928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/1db1bf9bea374f37b124738f1edc4d39/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.139572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, user_id=1db1bf9bea374f37b124738f1edc4d39, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.141893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062e754-60fc-44d6-bf1c-95b323cbb4dd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:54.205833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f956-28cb-40e8-acba-dba135c415d7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.227440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 69/135] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:26:53 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/1db1bf9bea374f37b124738f1edc4d39/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:54.246595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87d7-5aeb-4365-81bb-b8be85ff7d44 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.246994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87d7-5aeb-4365-81bb-b8be85ff7d44 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.247453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87d7-5aeb-4365-81bb-b8be85ff7d44 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.249063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.249456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 70/136] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:54.250461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.250937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.251375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] PATH_INFO: `/v3/projects/c619f6c91b6345d8a519f24f4b3e3368` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.252894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c619f6c91b6345d8a519f24f4b3e3368)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.254821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:54.255496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.255909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.256281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.257808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 67/137] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:54 2022] GET /identity/v3/projects/c619f6c91b6345d8a519f24f4b3e3368 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:54.264674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:54.285790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:54.285790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:54.373514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:54.444962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 71/138] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:54.468926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88022fb5-cada-4490-871e-453f8a4c20c5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.471917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:54.482325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:54.528894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.529998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.530382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.530780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] PATH_INFO: `/v3/projects/c619f6c91b6345d8a519f24f4b3e3368/users/ed5bda805a18497ab72e318c7b4698e5/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.533082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] RBAC: Authorizing `identity:create_grant(project_id=c619f6c91b6345d8a519f24f4b3e3368, user_id=ed5bda805a18497ab72e318c7b4698e5, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.534695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:54.604820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 68/139] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:26:54 2022] PUT /identity/v3/projects/c619f6c91b6345d8a519f24f4b3e3368/users/ed5bda805a18497ab72e318c7b4698e5/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:54.650724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dfc6a-d129-4c72-85b0-8175d44f7ce9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.696359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.697378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.697761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.698136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.699634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.701194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d593fe-4fd9-4c9f-8ac7-730683f646d7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:54.702985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 72/140] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:54.708667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:54.832255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17ada6-f4ad-4d0e-9216-d19cb04f7573 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:54.877224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:54.877932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:54.878216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:54.878494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:54.879272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:54.880785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7773f0-645e-4f29-a4a3-d01a91a6b951 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:55.033556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 69/141] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:26:54 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:26:56.480628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49482cd2-df31-46b1-b53f-4da01fdf5a46 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:56.481288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49482cd2-df31-46b1-b53f-4da01fdf5a46 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:56.481288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49482cd2-df31-46b1-b53f-4da01fdf5a46 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:56.482521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 73/142] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:56.489568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:56.490074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:56.490146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:56.509856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:56.512095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:56.535875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:56.605541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 70/143] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:56.654228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49482cd2-df31-46b1-b53f-4da01fdf5a46 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:56.738893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:56.799427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49482cd2-df31-46b1-b53f-4da01fdf5a46 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:56.852904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:56.854199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:56.854712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:56.855636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:56.868940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-93b830ba-c341-4695-b6f2-1c78429a4f05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:56.877924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:56.878977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:56.879765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 74/144] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:26:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:56.887399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:56.925325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5da93-8919-459c-91ec-ad3e2287afb9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:56.925628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5da93-8919-459c-91ec-ad3e2287afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:56.925881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5da93-8919-459c-91ec-ad3e2287afb9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:56.927452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 75/145] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:56.935346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:56.935652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:56.935903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:56.967568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:56.968856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:57.106257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252a71-ecab-4c05-92e3-bd09bd491c9f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:57.147791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 76/146] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:57.177300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:57.178080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.179333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:57.182284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.182612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.188600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:57.188917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:57.192099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:57.218723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 71/147] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:26:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:57.230419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:57.256694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:57.281859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:57.394902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5bdc9-5e64-4a14-a0b5-825f7ef97433 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:57.427408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899220-8576-430f-a87f-5f19ae584d82 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:57.458183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:57.459015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.459297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.459574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.476747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:57.477744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.478029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.478307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.479595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:57.480568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:57.481298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd01ac-f64f-4d9c-8522-35be79028273 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:57.485647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 72/148] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:57 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:26:57.487032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f56d-7bb7-44ab-a91f-7e804b50c323 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.487319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f56d-7bb7-44ab-a91f-7e804b50c323 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.487599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f56d-7bb7-44ab-a91f-7e804b50c323 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.489515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 73/149] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:26:57.494503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.495151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.495151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:57.498774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:57.500299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 77/150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:26:57.501892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:57.523430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:57.524654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:57.653931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70d16-245b-401c-913f-99e306d98acc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:57.657544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:57.680068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 74/151] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:57.680971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:57.710581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:57.711601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:57.711888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:57.712164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:57.788828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:57.903645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:57.905240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:58.036819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cce9-308d-493b-a97a-fa9f5325c518 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:58.111440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:58.121230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 78/152] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:26:58.133413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:58.137326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:58.137326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.137326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.137326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.137703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:58.137999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:58.139324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:58.155026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 75/153] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:58.155925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:58.366382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66afa2c-f964-4b86-8e74-5d4724d7624d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:58.374455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:58.414300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd298fe-f949-47b9-a8a4-27968a98efad None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:58.456770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:58.458510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.459476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:58.460602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.461086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.461573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.462044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.463057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.464828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:58.468121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:58.473242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 76/154] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:58 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:58.474523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:58.490189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4defd028-e7d7-425f-a242-c1ef274b6471 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:58.499800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:58.507006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:58.507822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:58.513051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 79/155] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:58.517623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244955f-f2aa-4591-8cdb-34a6af4700a1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.519444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244955f-f2aa-4591-8cdb-34a6af4700a1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.519927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244955f-f2aa-4591-8cdb-34a6af4700a1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.521584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 80/156] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:58.526223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c81dda-40c1-4a77-b57b-557227ab0dca None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.526794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c81dda-40c1-4a77-b57b-557227ab0dca None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.529302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c81dda-40c1-4a77-b57b-557227ab0dca None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.531077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 81/157] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:58.533843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c81dda-40c1-4a77-b57b-557227ab0dca None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:58.647360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825d2f-75c7-479b-85be-fc0abfb208e9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:58.660614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:58.690956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:58.692302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.692817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.694357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] PATH_INFO: `/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.702009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] RBAC: Authorizing `identity:get_user(user_id=f34b5a5061d84b4e92e9cf33bf93cb21)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:58.703932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:58.706029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 77/158] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:58 2022] GET /identity/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:58.707159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:58.726867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:58.768218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c81dda-40c1-4a77-b57b-557227ab0dca None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:58.854793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401604-a116-4a35-a69e-47838eee886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:58.854793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401604-a116-4a35-a69e-47838eee886d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.854793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401604-a116-4a35-a69e-47838eee886d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.854793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401604-a116-4a35-a69e-47838eee886d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.855191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401604-a116-4a35-a69e-47838eee886d None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:58.855191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401604-a116-4a35-a69e-47838eee886d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:58.856113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401604-a116-4a35-a69e-47838eee886d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:58.866660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 82/159] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:26:58 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:58.867970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:58.868261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:58.868543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:58.922571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:58.923841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:58.956739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0a82-9131-40bf-96a5-0e49d5be5eac None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.055638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.056421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.056756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.057042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.057837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:26:59.058693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:59.060827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad575a7-8190-463f-bc9f-c787a763c46c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:59.078644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 78/160] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:59.080754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.081245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.081740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.103189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:26:59.104817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:26:59.136283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 83/161] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:59.137578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.205336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 79/162] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:26:59.205931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:59.215652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.291142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:59.375782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278c99-5a54-4a74-a85d-de4b2498b193 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.423837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a82a5-ab24-42cb-98a0-5ddd0d3938c3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.434495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.435946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.436427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.436898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.438435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:59.440179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:59.442093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 80/163] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:59.443151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.477907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.478742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.482995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.483278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.484724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:59.486370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:59.488709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 84/164] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:26:59.495308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.536727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:59.576453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c362-83ec-4568-b9ec-fd220cbe7ec9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.614015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa78671-91e7-484b-8868-c281f15c3038 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.616909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.617867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.618297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.618736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.631791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:59.640447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a634dc55-412c-4eec-9080-60d54542c409 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:59.641342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:59.642379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:26:59.643333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:26:59.645525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 81/165] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:59.647329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac430-ff2e-4412-869b-c415d55eefaa None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.647817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac430-ff2e-4412-869b-c415d55eefaa None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.648690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac430-ff2e-4412-869b-c415d55eefaa None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.650212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 82/166] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:26:59.651385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac430-ff2e-4412-869b-c415d55eefaa None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.657128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.657943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.658220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.658493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.674082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:26:59.688184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:26:59.689158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 85/167] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:26:59.690385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:26:59.793802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac430-ff2e-4412-869b-c415d55eefaa None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.823462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ea08e-de6e-4c71-9ad1-07983795e1f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:26:59.826291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:26:59.835952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.837003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.837481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.838008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.862491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:26:59.863423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:26:59.863704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:26:59.863977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/f34b5a5061d84b4e92e9cf33bf93cb21/roles/ca074f0bc20f4f9aaee65f81d2bcd83d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:26:59.866335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, user_id=f34b5a5061d84b4e92e9cf33bf93cb21, role_id=ca074f0bc20f4f9aaee65f81d2bcd83d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:59.867678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:59.919079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:26:59.920930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ad699-272a-4cd6-bf72-906fdeab1ee4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:26:59.939255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 86/168] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:26:59 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/users/f34b5a5061d84b4e92e9cf33bf93cb21/roles/ca074f0bc20f4f9aaee65f81d2bcd83d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:26:59.940688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.018959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.095694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933cfb-dd53-47f9-bb6d-616e23dbf5c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.105796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 83/169] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:00.108608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27356b-14bf-4699-b297-8aafc52ea722 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.112372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27356b-14bf-4699-b297-8aafc52ea722 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.112372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27356b-14bf-4699-b297-8aafc52ea722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.134682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-af27356b-14bf-4699-b297-8aafc52ea722 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:00.135910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-af27356b-14bf-4699-b297-8aafc52ea722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:00.150401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.151398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.151682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.152013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.152909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:00.153188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.154710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.169980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 87/170] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:26:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.174674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.175755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.239286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 84/171] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:00.244091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168efcb-4028-4fdb-b0c6-517b247a9ddf None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.244380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168efcb-4028-4fdb-b0c6-517b247a9ddf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.244794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168efcb-4028-4fdb-b0c6-517b247a9ddf None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.252543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 85/172] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:00.254043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b567aa-fb29-42ff-9402-4188a8b39bf2 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.254327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b567aa-fb29-42ff-9402-4188a8b39bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.254616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b567aa-fb29-42ff-9402-4188a8b39bf2 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.256089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 86/173] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:00.256828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b567aa-fb29-42ff-9402-4188a8b39bf2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.316739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841e13b-9025-4544-b9d3-5da22d9dab47 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.319570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.361243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.361827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.362103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.362378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.363147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:00.363453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.364870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.375739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 88/174] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.376590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.392083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b567aa-fb29-42ff-9402-4188a8b39bf2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.434667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.435549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.435831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.436179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.437443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.439132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25913566-9bc0-4928-9523-d0028af286a3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.445683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 87/175] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.447329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40503b49-88c0-4be9-8c8d-a5c274d5ed2e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.447620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40503b49-88c0-4be9-8c8d-a5c274d5ed2e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.447903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40503b49-88c0-4be9-8c8d-a5c274d5ed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.466389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-40503b49-88c0-4be9-8c8d-a5c274d5ed2e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:00.467636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-40503b49-88c0-4be9-8c8d-a5c274d5ed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:00.492308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.509369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e4bb-ca6c-44e5-a352-49a308e267a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.550566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 88/176] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:00.550566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.550566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.550566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.556119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.557046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.557328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.557807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] PATH_INFO: `/v3/users/ed5bda805a18497ab72e318c7b4698e5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.559767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5bda805a18497ab72e318c7b4698e5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.561240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.567352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 89/177] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/users/ed5bda805a18497ab72e318c7b4698e5 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.567352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.568226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:00.570036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:00.602186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.650493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 89/178] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:00.653754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.694944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600eab-4aac-47b0-a89c-dba2f75208d8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.743964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.744769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.745050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.745325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.758198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:00.766519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:00.767519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:00.768294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 90/179] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:00.769743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.770348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.789757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920eeea-43f7-4cd6-81f8-37e31ad7da47 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.829396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.830280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.830588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.830873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] PATH_INFO: `/v3/users/1db1bf9bea374f37b124738f1edc4d39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.832080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] RBAC: Authorizing `identity:get_user(user_id=1db1bf9bea374f37b124738f1edc4d39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.833528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.839750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 90/180] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/users/1db1bf9bea374f37b124738f1edc4d39 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.839750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.896790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:00.909951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283d46-3d16-4a46-990e-95446c694976 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:00.951148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:00.951979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:00.952638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:00.952931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:00.954073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:00.954388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:00.956397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:00.963119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 91/181] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=a0903d733c974f51970116b3948f5ff8 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:00.963785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:00.997195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aadb-e94c-4214-8383-9469f9960c23 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.049014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.050345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.050811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.051094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.068559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:01.081321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:01.082535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:01.083550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:01.086738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 91/182] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:01.087891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.116662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.136917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86731-c8cf-441e-9568-2447130d0f64 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.184915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.186238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.186712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.187105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.228320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.244970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564b47e-605c-4813-85b3-a15dc3bc9e25 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.288319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.289113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.289392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.289660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.290952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.293200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.294271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.295156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 92/183] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:01.295747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.299582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.423033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.432027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 92/184] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:01.433048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.459923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81298dd7-556e-4b5a-96a5-c65c9b7f77e1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.517737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.518444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.518688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.519541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.519541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.520882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:01.521368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.523824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.537528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 93/185] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:01.538897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.584138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867989d-be00-4c19-8886-140d253d6146 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.623591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.624258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.624542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.624815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.661838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0be9b-0038-4ab6-9498-cfcc1373ddd4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.706235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.706865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.707147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.707422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.708177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:01.708485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.709869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.713159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.715368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.720711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 94/186] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:01.721405 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.734877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.782569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:01.836065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201288b2-d563-41c5-8eb5-eb8e38abed15 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:01.837066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 93/187] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:01.838291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:01.880648 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:01.881358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:01.881645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:01.881923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:01.882737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:01.884505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:01.967317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc117-f5fd-4431-b992-85f9799f16cf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.004296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 95/188] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:01 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:02.005349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.020384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.021719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.022250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.022739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] PATH_INFO: `/v3/projects/c619f6c91b6345d8a519f24f4b3e3368` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.024668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] RBAC: Authorizing `identity:get_project(project_id=c619f6c91b6345d8a519f24f4b3e3368)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.026597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.034464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 94/189] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:01 2022] GET /identity/v3/projects/c619f6c91b6345d8a519f24f4b3e3368 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:02.034464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.076172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.102719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.150696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7337-dadb-4b3d-a3a1-aa527f695013 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.180339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5aaf-8888-4d52-b766-d3f78f644d5f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.199387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.200972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.201620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.202120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.223143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.224547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.226474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 96/190] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:02.227443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.231141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.232523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.232974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.233427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.235939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.237770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.245060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 95/191] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:02.245060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.369735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.372349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49931b-32da-418a-9560-7cb4497df399 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.382232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5451527-ebd8-4591-862c-6302a01b74de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.410569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.428538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.429695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.429979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.430184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.430319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.430864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.431148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.431427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.431525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.432269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=c619f6c91b6345d8a519f24f4b3e3368, user.id=ed5bda805a18497ab72e318c7b4698e5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:02.432584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.432909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8ad4-eb8a-4adf-af73-f03ef4d4919d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.434906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.435627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 97/192] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:02.437586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ce16-1214-43e8-827a-4c395407fbbd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.438127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ce16-1214-43e8-827a-4c395407fbbd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.438582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ce16-1214-43e8-827a-4c395407fbbd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.439913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 98/193] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:02.440639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ce16-1214-43e8-827a-4c395407fbbd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.451970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 96/194] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/role_assignments?user.id=ed5bda805a18497ab72e318c7b4698e5&scope.project.id=c619f6c91b6345d8a519f24f4b3e3368&role.id=a0903d733c974f51970116b3948f5ff8 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:02.452949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.604286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f862b64-1878-423a-99ce-cf0ac43ce629 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.654580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.655030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.655311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.655587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.662895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ce16-1214-43e8-827a-4c395407fbbd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:02.690056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.707023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:02.711375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.711375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.711375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.740907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.742698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.782469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.815429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:02.817001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6386ea-5076-4520-87c0-c17793fe6c7e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:02.875433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 97/195] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:02.876173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:02.940210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 99/196] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:02.950132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:02.951506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:02.952109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:02.953486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:02.977623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:02.979191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:03.002367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398ab1-3953-4b4b-a986-bf912cecb934 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.042203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.043812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.043964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.044193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.045014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=1db1bf9bea374f37b124738f1edc4d39) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:03.045324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.046826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.053890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 98/197] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:02 2022] GET /identity/v3/role_assignments?user.id=1db1bf9bea374f37b124738f1edc4d39&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.054752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.074939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 100/198] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:03.076252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.099412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:03.178882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08a814-18a2-4517-820c-f2a2e870585e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.205262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddbfee-13b5-4208-a23d-f6ff3824fdf3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.209669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:03.225823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.226764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.227214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.232124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.232124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.232124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.233818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 99/199] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.234451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.251068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.252797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.253326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.253838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.255683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.257741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.261102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 101/200] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.262934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.364617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617e3c-4e00-4617-89a5-6500fcee2fe8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.393360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:03.406054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fa498-c265-4164-9986-3480cb847917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.407230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa498-c265-4164-9986-3480cb847917 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.407733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa498-c265-4164-9986-3480cb847917 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.408203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa498-c265-4164-9986-3480cb847917 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.416609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1661c-25f5-495b-a1e9-c9716720dbd2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.420365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fa498-c265-4164-9986-3480cb847917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-255fa498-c265-4164-9986-3480cb847917 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:03.428591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:03.429559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:03.430455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:03.433883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 100/201] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:03.435095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fa498-c265-4164-9986-3480cb847917 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.458398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.459120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.459402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.459677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.460590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:03.460904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.462999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.470565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 102/202] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/role_assignments?user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.471577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.528274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:03.607054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fa498-c265-4164-9986-3480cb847917 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.641419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92a5b-6d3a-423e-89b7-7e08734dc76c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.657442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.658459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.658976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.659451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.660530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:03.661044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.662784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.672153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 101/203] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/role_assignments?user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.703438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.704914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.705424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.705919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.706966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:03.707595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.709279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.727817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 103/204] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.736566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.882008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ea4d6-101e-4b64-b8ac-4ca55be46acb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:03.901352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:03.943266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:03.944252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:03.944538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:03.944815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] PATH_INFO: `/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:03.946085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] RBAC: Authorizing `identity:get_user(user_id=f34b5a5061d84b4e92e9cf33bf93cb21)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:03.947599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:03.949561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 102/205] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/users/f34b5a5061d84b4e92e9cf33bf93cb21 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:03.955612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:03.977373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:04.090144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba219a-f538-4a76-ae37-ee510475d035 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:04.134057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:04.134756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:04.135040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:04.135318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:04.221640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:04.231410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:04.246274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:04.392891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 104/206] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:04.398525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:04.571004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb724-90c6-4b77-b67b-0514201288d0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:04.624909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:04.625784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:04.626069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:04.626349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:04.631347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:04.633238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec8451-b01c-426b-894a-dab0c8966ae8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:04.635754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 103/207] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:04 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:04.643139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:04.803670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18aa90-2a1e-4eab-b621-afb0c96640d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:04.823177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:04.853619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:04.854294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:04.854653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:04.855555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:04.856573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, scope.project.id=fc9b385b87ca4f5aab27028f16841078, user.id=f34b5a5061d84b4e92e9cf33bf93cb21) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:04.856891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:04.858359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6661-054f-41d2-9987-016d91839056 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:04.862421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97a7e-ef28-4c07-9bd1-3ec11e9db904 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:04.862940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97a7e-ef28-4c07-9bd1-3ec11e9db904 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:04.863415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97a7e-ef28-4c07-9bd1-3ec11e9db904 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:04.864923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 105/208] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:04 2022] GET /identity/v3/role_assignments?user.id=f34b5a5061d84b4e92e9cf33bf93cb21&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:04.870576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 104/209] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:04.876227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9657-23a3-4ba6-a02d-69190ac79166 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:04.876736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9657-23a3-4ba6-a02d-69190ac79166 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:04.877161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9657-23a3-4ba6-a02d-69190ac79166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:04.904690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fe3f9657-23a3-4ba6-a02d-69190ac79166 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:04.906134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fe3f9657-23a3-4ba6-a02d-69190ac79166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:05.014159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 106/210] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:05.048131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f97a7e-ef28-4c07-9bd1-3ec11e9db904 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.104158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:05.165852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f97a7e-ef28-4c07-9bd1-3ec11e9db904 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.193161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e121b2-0062-48ed-a0ab-f839cc122a29 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.193639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e121b2-0062-48ed-a0ab-f839cc122a29 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.194083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e121b2-0062-48ed-a0ab-f839cc122a29 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.196919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 107/211] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:05.204901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.205410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.205942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.213698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.214734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.215202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.215485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.227282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:05.236285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:05.237569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:05.238373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:05.242690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 105/212] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:05.244051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.245003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:05.247685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:05.354803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7d63-d89c-4783-a9e7-0d2033789230 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.365343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:05.374167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 108/213] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:05.395295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.396073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.396449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.396739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.397514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:05.397818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:05.399310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:05.411260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.414742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 106/214] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:05.422575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.480963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:05.569766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d83b5-538b-4601-b701-4eda2c18c584 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.632825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce992e4-9320-470d-820d-68e7c18b4ce3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.643343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.644883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.645178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.645463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] PATH_INFO: `/v3/users/591f69135f614f759aed20e203122cc4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.665471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=591f69135f614f759aed20e203122cc4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:05.668283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:05.674632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 107/215] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:05 2022] GET /identity/v3/users/591f69135f614f759aed20e203122cc4 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:05.677339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.686770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.687983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.688468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.689006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.690531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:05.692248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:05.694478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 109/216] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:05.703575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:05.720157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:05.818346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c2a2-a56d-4a5a-aca1-05de40137bb4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.821024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3fbbc-bc97-4282-a1f7-d0a5ceb3348a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:05.824935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:05.860560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.860785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:05.861556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.861837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.862113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.862349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:05.862863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:05.862962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:05.863410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:05.864479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:05.945992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 110/217] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:05 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:27:05.962929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:05.965646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:06.004101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:06.102683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 108/218] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:06.106735 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:06.232385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ebee3-2ec2-4963-996c-bc50935e85e1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:06.275129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:06.276446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.277394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.277868 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.279747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:06.281565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89456a12-f109-4ed4-bd70-2334f21b507f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:06.283961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 111/219] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:06 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:06.293293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:06.366164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5762d-fcdd-4ba9-9c66-da429d2172d0 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.366841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5762d-fcdd-4ba9-9c66-da429d2172d0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.367323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5762d-fcdd-4ba9-9c66-da429d2172d0 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.369343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 112/220] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:06.377448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415c6a-2344-4002-9581-ba7f3022e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.378791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415c6a-2344-4002-9581-ba7f3022e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.379304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415c6a-2344-4002-9581-ba7f3022e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.410513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fc415c6a-2344-4002-9581-ba7f3022e0d0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:06.419306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fc415c6a-2344-4002-9581-ba7f3022e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:06.462011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:06.468847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a6df5-6729-4284-93d5-c5d592413c07 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:06.524273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:06.563333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:06.564112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.564401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.564686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.565544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=591f69135f614f759aed20e203122cc4) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:06.566039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:06.568282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704457-a492-4fbb-8c84-64fccf9e893f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:06.574683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 113/221] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:06.581603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc173731-2139-4de5-b3fe-ef4f28c0eee2 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.582101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc173731-2139-4de5-b3fe-ef4f28c0eee2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.582534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc173731-2139-4de5-b3fe-ef4f28c0eee2 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.584509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 114/222] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:06.586593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 109/223] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:06 2022] GET /identity/v3/role_assignments?user.id=591f69135f614f759aed20e203122cc4&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=a0903d733c974f51970116b3948f5ff8 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:06.594751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.595039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.595321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.607711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc173731-2139-4de5-b3fe-ef4f28c0eee2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:06.618649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:06.619960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:06.772670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 110/224] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:06.813812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc173731-2139-4de5-b3fe-ef4f28c0eee2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:06.837301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:06.881516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:06.883673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:06.884340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:06.884858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:06.903872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:06.913926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:06.915040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:06.916001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:06.919636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 115/225] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:06.926970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:06.962680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:07.070908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22a6d7-30f6-43b0-91ab-e8ee3674471a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.126573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:07.148198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.149372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.149888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.150351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.153091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b2471-7531-46c3-a372-8b3b161a1beb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.174023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:07.183233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:07.184467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:07.185475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:07.192704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 111/226] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:07.213454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:07.222951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.223891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.224362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.224777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.225787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:07.226244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:07.228181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:07.239186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 116/227] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:07.245718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:07.442918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:07.463444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853422-bf84-4e43-8f5a-99b5ba9b014d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.496573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4b571-fafc-4de8-8312-4ea7faa63164 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.502776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.504740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.505024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.505305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.506614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:07.508208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:07.510278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 117/228] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:07 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:07.519532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:07.559343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.560050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.560338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.560616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.561381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:07.561688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:07.563119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:07.582787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 112/229] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:07.599647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:07.648493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8dbb5-d854-46b6-a6ce-187981e90424 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.686404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.687319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.687609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.688343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.689625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:07.691136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:07.693287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 118/230] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:07 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:07.700024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:07.739147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:07.798942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:07.830823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7292d-c223-4171-8b1b-7f60d633a702 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.871907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:07.873113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:07.873404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:07.873740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:07.877582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a, role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:07.879148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:07.927627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dc269-81d7-4730-a947-9e9db51b51c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:07.947544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 119/231] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:07 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:08.007455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:08.008393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:08.008678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:08.008955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:08.010382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:08.012021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:08.014172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 113/232] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:07 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:08.029896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:08.150972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:08.202767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9998cd0-6d24-442e-b8d2-1b3d905e4d56 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:08.251423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:08.252307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:08.252593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:08.252873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:08.254077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:08.255571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4138-fb78-4d06-ac3e-b19788d24433 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:08.257552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 120/233] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:08 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:08.267719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:08.401095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023875b7-bf92-4705-9fed-f615de17c1f0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:08.416310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:08.443732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:08.444643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:08.444924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:08.445197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:08.447527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:08.449641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a465195-a8f4-4ca1-828b-f2a6292052c5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:08.519275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 114/234] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:08 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:08.894637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288720-b42d-4267-8f0c-29b41851a5f9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:08.894943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288720-b42d-4267-8f0c-29b41851a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:08.895129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288720-b42d-4267-8f0c-29b41851a5f9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:08.896867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 121/235] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:08.902777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3c62-57c6-4fbb-8807-0f881f2da313 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:08.903373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3c62-57c6-4fbb-8807-0f881f2da313 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:08.903875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3c62-57c6-4fbb-8807-0f881f2da313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:08.923628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-14fc3c62-57c6-4fbb-8807-0f881f2da313 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:08.925244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-14fc3c62-57c6-4fbb-8807-0f881f2da313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:09.014195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 115/236] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:09.060646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a288720-b42d-4267-8f0c-29b41851a5f9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.193459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2098-9858-417f-9d9c-63e074588235 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.193791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2098-9858-417f-9d9c-63e074588235 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.194028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2098-9858-417f-9d9c-63e074588235 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.195533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 116/237] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:09.202060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e436-87f9-476f-82fd-ce2ab0181273 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.202721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e436-87f9-476f-82fd-ce2ab0181273 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.203198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e436-87f9-476f-82fd-ce2ab0181273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.223143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a288720-b42d-4267-8f0c-29b41851a5f9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:09.229848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4a07e436-87f9-476f-82fd-ce2ab0181273 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:09.236950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4a07e436-87f9-476f-82fd-ce2ab0181273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:09.278762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:09.279957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.280734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.282181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.283713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:09.285398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1725c3-124a-4005-ad71-1d16a898e626 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:09.287958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 122/238] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:09.290017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cb42b-b926-4a5b-b1dd-093858d21d63 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.290516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cb42b-b926-4a5b-b1dd-093858d21d63 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.291483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cb42b-b926-4a5b-b1dd-093858d21d63 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.294538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 123/239] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:09.295535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cb42b-b926-4a5b-b1dd-093858d21d63 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.318294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:09.321651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 117/240] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:09.326946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.326946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.326946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.356734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:09.359031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:09.401769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:09.453146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 118/241] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:09.454016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.460300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cb42b-b926-4a5b-b1dd-093858d21d63 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:09.511023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:09.512584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.517106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.517106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.517106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:09.517106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:09.575543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f935e14-2b74-4365-9c07-23d1566e1d72 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:09.621840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:09.622938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.623219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.623495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.630716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 124/242] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:09 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:09.632025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.637017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:09.645436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:09.646405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:09.647229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 119/243] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:09.652912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.733257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:09.733569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:09.777900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd16a4-2162-46c3-bf0d-0bf74a460621 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:09.821966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:09.822968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.823257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.823603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.837895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:09.846940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:09.849078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:09.851029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 125/244] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:09.856360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.923551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6c17-43ff-4205-b2ca-582f2bb98336 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:09.966779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:09.967923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:09.968419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:09.968881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:09.969820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:09.970314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:09.971924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:09.982242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 120/245] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:09.992424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:09.994835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f2c6f-9166-48d6-9e8e-82160e74c579 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.034472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:10.048600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.049632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.050138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.050638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.051721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:10.052235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.053863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.065634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 126/246] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:10.073162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:10.127508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:10.147859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145164f-821e-4cc1-8839-607f6c3c2bc5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.203206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.204275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.204728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.205152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] PATH_INFO: `/v3/users/591f69135f614f759aed20e203122cc4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.207532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] RBAC: Authorizing `identity:get_user(user_id=591f69135f614f759aed20e203122cc4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.209167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.211980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 121/247] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:09 2022] GET /identity/v3/users/591f69135f614f759aed20e203122cc4 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:10.220265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:10.252776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad623-66dd-489b-979c-f1f6dd36a965 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.292561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.293802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.294261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.294738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] PATH_INFO: `/v3/users/5f1071b5409143c3acbbbef219c47539` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.311742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1071b5409143c3acbbbef219c47539)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.313745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.316379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 127/248] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:10 2022] GET /identity/v3/users/5f1071b5409143c3acbbbef219c47539 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:10.327684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:10.348963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63131036-f4eb-457c-bcc9-5dc62dd54719 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.389962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.390923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.391214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.391729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.393160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.394980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.397088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 122/249] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:10 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:10.403826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:10.422582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:10.434900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:10.526822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060bca1-4a66-4b7f-b753-268a414bae17 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.532932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548b745-bfcd-4e3a-8ab5-6b5f686d4d40 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:10.570023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.571298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.571862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.572416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/591f69135f614f759aed20e203122cc4/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.574836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, user_id=591f69135f614f759aed20e203122cc4, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.576681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92778b76-53b6-429b-802b-19a5edd91c83 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.607109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:10.607786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.608068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.608340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.652881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 123/250] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:10 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/591f69135f614f759aed20e203122cc4/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:27:10.722678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:10.724162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:10.733667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2379cc0-4462-48e9-a348-62d1c9be99be None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.733953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2379cc0-4462-48e9-a348-62d1c9be99be None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.734235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2379cc0-4462-48e9-a348-62d1c9be99be None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.736040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 124/251] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:10.741924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:10.742332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:10.742750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:10.764041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:10.765410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:10.804702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:10.889555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 128/252] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:10.895434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:10.941238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 125/253] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:10.977129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:11.001882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:11.012003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f02079-5eb2-4ee1-8d84-eb7e7c73cc80 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:11.049720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:11.050941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.051392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.051807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.053319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:11.055032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:11.056934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 129/254] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:10 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:11.063714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:11.096710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:11.185116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bf0e9-d0e6-4429-8cbd-c9976468766a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:11.227363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:11.228037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.228497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.228777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.230632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=5f1071b5409143c3acbbbef219c47539) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:11.230976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:11.232872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:11.240039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 130/255] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:11 2022] GET /identity/v3/role_assignments?user.id=5f1071b5409143c3acbbbef219c47539&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:11.253845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add46f8a-41e6-4bf8-8d0a-ab9f4bd74c6c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:11.349577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:11.351254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.351760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.352231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.375779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:11.384651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:11.385609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:11.386660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:11.390664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 126/256] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:11.392604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:11.508716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:11.536684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6837a5e-155e-4c97-822f-fbb565dd527b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:11.592016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:11.592716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.593063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.593341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.594168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:11.594480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:11.596076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8d9e-7529-4334-85fe-67ba51e39de4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:11.610809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 131/257] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:11.615079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:11.737629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:11.765709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba59733-1d29-4e8f-907c-acaf739eca14 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.766095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba59733-1d29-4e8f-907c-acaf739eca14 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.766652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba59733-1d29-4e8f-907c-acaf739eca14 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.768317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 132/258] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:11.777250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2628d3-e991-4924-900c-d95467768784 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.777673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2628d3-e991-4924-900c-d95467768784 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.778044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2628d3-e991-4924-900c-d95467768784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.789209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9966-7471-4d92-b796-f003a86b372d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:11.802225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4b2628d3-e991-4924-900c-d95467768784 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:11.803627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4b2628d3-e991-4924-900c-d95467768784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:11.853870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:11.855922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:11.856363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:11.856643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:11.863611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:11.865194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:11.867444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 127/259] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:11 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:11.876147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:11.881504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 133/260] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:11.906888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2628d3-e991-4924-900c-d95467768784 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.065625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:12.078053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361af3-2f60-4872-9dcd-bf3a1f5d8d23 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:12.099673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2628d3-e991-4924-900c-d95467768784 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:12.143505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:12.144368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:12.145367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:12.146200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:12.146681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:12.147136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:12.147560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:12.147976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:12.159976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:12.177451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:12.188386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:12.189384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:12.190194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:12.202026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 134/261] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:12.211627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.289923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:12.293671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:12.414163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fd7fa-8c82-4a22-a9c8-a3563f55858b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:12.487039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:12.497716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:12.498769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:12.499213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:12.499651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:12.500583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:12.501688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:12.503484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:12.506149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 128/262] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:12.513092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.535762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 135/263] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:12.541591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.603981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:12.676957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ad602-19bd-4619-9d2e-af70f826b47e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:12.714328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:12.715760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:12.716251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:12.716722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:12.718190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:12.719943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:12.721877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 129/264] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:12 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:12.728236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.754411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa7b25-f949-4e43-873f-115f83eafdc5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:12.802880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:12.804230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:12.804712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:12.805247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] PATH_INFO: `/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:12.809578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] RBAC: Authorizing `identity:get_user(user_id=3aca8cf0f5fd4b7683dee6bac4c3ac5a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:12.812733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:12.816175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 136/265] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:12 2022] GET /identity/v3/users/3aca8cf0f5fd4b7683dee6bac4c3ac5a => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:12.826219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:12.946397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:13.036969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c05cf5-1d1c-4543-91ca-d55c62d966ad None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:13.045242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725b97-b6af-4200-94c2-a25086e83f2f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:13.058230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:13.087186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:13.088207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.088850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.090066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.136100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:13.136820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.137292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.137613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.138464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:13.138804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:13.140266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:13.159231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 130/266] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:12 2022] GET /identity/v3/role_assignments?user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 608 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:13.258672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:13.261048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f969d9-dbd5-4b35-925b-cb0b752b1f6f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:13.394927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:13.450505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 137/267] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:13.456579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:13.470455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50094-73ac-4329-a071-8fb48d019767 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.470815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50094-73ac-4329-a071-8fb48d019767 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.471049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50094-73ac-4329-a071-8fb48d019767 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.473141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 138/268] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:13.488709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dffc1-51c7-466a-b0de-a7c1d4f897a8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.488989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dffc1-51c7-466a-b0de-a7c1d4f897a8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.489456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dffc1-51c7-466a-b0de-a7c1d4f897a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.517406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ff8dffc1-51c7-466a-b0de-a7c1d4f897a8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:13.518839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ff8dffc1-51c7-466a-b0de-a7c1d4f897a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:13.591666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d068-751d-4ad9-b6b1-c7d4d0fe8731 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:13.629335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 139/269] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:13.638286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73f82-afeb-4119-b9cd-b96f510ba146 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.640407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73f82-afeb-4119-b9cd-b96f510ba146 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.640709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73f82-afeb-4119-b9cd-b96f510ba146 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.643824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:13.644709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.644989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.645263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.646105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 140/270] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:13 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:13.646829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:13.649560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.649839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.650113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.652680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:13.660001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 131/271] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:13 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:13.660900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:13.682661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:13.685043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:13.732217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:13.784158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 141/272] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:13.785413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:13.815287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19225-6aab-425b-9728-3cbf11968ed2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:13.820914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:13.855721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:13.856832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.857301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.857783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.869556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:13.878700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:13.879646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:13.880472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:13.883474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 132/273] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:13.884585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:13.923274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09d834-80f5-46ec-a3a4-2a14d154cbd4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:13.961559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:13.962229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:13.962507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:13.962807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:13.963643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:13.963953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:13.965856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:13.979091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 142/274] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:13 2022] GET /identity/v3/role_assignments?user.id=3aca8cf0f5fd4b7683dee6bac4c3ac5a&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:13.979756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.012471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0600-4523-4c88-b8e8-a4416b34a9a8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.055074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.067651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.069659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.070436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.071272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.076356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:14.083806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:14.085956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 133/275] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:14.092439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.234380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.253169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43eb56-a42f-4c3c-9be9-cb0d398e0c9b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.255548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91135-bbd3-4a26-b47b-440042cff958 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.299883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.300565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.300894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.301173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.301358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.301879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:14.302819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.303303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.303421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaec889-2568-47d0-bd54-95c35cf5fbc4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:14.303881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.304836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:14.305939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:14.309153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:14.322581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 143/276] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:14.327681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.428025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 134/277] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:14 2022] POST /identity/v3/users => generated 293 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:27:14.429339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de910f-bb0d-4b9a-bdf3-2db6eaf4b98a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.429676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de910f-bb0d-4b9a-bdf3-2db6eaf4b98a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.429895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de910f-bb0d-4b9a-bdf3-2db6eaf4b98a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.436934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 135/278] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:14.438477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24c6eb-a5a9-4d79-89e4-74c7ec984935 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.438810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24c6eb-a5a9-4d79-89e4-74c7ec984935 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.439093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24c6eb-a5a9-4d79-89e4-74c7ec984935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.457361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7d24c6eb-a5a9-4d79-89e4-74c7ec984935 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:14.458615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7d24c6eb-a5a9-4d79-89e4-74c7ec984935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:14.459237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cfe60-45c9-4b03-9bf7-709d264a4df2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.481555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.511987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.513110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.513473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.514703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] PATH_INFO: `/v3/users/b2635acf532a4461b3382fab26038760` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.531697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] RBAC: Authorizing `identity:get_user(user_id=b2635acf532a4461b3382fab26038760)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:14.533594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:14.535677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 144/279] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:14 2022] GET /identity/v3/users/b2635acf532a4461b3382fab26038760 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:14.542942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.553479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.568131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 136/280] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:14.570701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18b7e0-1b8b-4249-9702-64156f8a7b91 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.571236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18b7e0-1b8b-4249-9702-64156f8a7b91 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.571692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18b7e0-1b8b-4249-9702-64156f8a7b91 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.574560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 137/281] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:14.586670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.586670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.586670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.603877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:14.605150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:14.670493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926c7dd-3759-459a-8b92-32e4a773297e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.703216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 138/282] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:14.703981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.724845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.725683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.726066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.726343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.727324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, user.id=b2635acf532a4461b3382fab26038760, scope.system=all) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:14.727633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:14.729103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:14.736166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 145/283] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 21 10:27:14 2022] GET /identity/v3/role_assignments?user.id=b2635acf532a4461b3382fab26038760&scope.system=all&role.id=a0903d733c974f51970116b3948f5ff8 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:14.737026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.818685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.832964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:14.847262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d9715-4869-4029-9460-8dda5c5cf343 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.890519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4fcd0-9fcd-4e3c-b1aa-9dec650bd45e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:14.900672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.901507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.901833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.902139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.919741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:14.928199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:14.929882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:14.931528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 139/284] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:14.937109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:14.969463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:14.971076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:14.971359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:14.973523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:14.998075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:15.006606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:15.007584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:15.008380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 146/285] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:15.014941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.067794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc0121-46ac-41aa-80ed-387f168d3a91 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.105263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.106490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.107019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.107519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.108613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:15.109130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.110787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.121492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 140/286] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:15.132891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.142284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:15.189735 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717545-648e-4d73-ad88-eec559d8a875 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.254931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:15.268018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.268715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.269000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.269275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.270041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:15.270349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.271787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.289258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b52f-3602-4bfb-9218-712b7d73c16e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.290731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 147/287] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:15.296436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.338869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.340566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.340953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.342208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] PATH_INFO: `/v3/users/591f69135f614f759aed20e203122cc4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.345358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] RBAC: Authorizing `identity:get_user(user_id=591f69135f614f759aed20e203122cc4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.350586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.358786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 141/288] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/users/591f69135f614f759aed20e203122cc4 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:15.365172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.493810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:15.525547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6ad3-ace6-4962-ad47-bd9d93206638 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.556835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f587f-3f5d-40a2-b322-63023960a13a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.574566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.574566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.574566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.574566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] PATH_INFO: `/v3/users/5f1071b5409143c3acbbbef219c47539` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.574933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] RBAC: Authorizing `identity:get_user(user_id=5f1071b5409143c3acbbbef219c47539)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.574933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.576505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 148/289] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/users/5f1071b5409143c3acbbbef219c47539 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:15.584005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.630667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.631978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.632472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.632936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.694112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:15.768734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.771081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.773936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aa164-d649-4f85-a6e3-bea052b463fb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:15.835806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:15.836679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:15.836957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:15.837229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:15.843330 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:15.844854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:15.847120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 149/290] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:15.852871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:15.906794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:15.961292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 142/291] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:15.966881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:16.056938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeafaa-1396-423a-882f-3fd799886beb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:16.110996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18fba2-c98d-46c2-a9c4-01ded00a7972 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:16.113670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:16.132699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:16.133607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:16.133886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:16.134160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/5f1071b5409143c3acbbbef219c47539/roles/ca074f0bc20f4f9aaee65f81d2bcd83d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:16.138913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, user_id=5f1071b5409143c3acbbbef219c47539, role_id=ca074f0bc20f4f9aaee65f81d2bcd83d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:16.140427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c255-4f32-4640-94c6-8db5348e89ce None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:16.157810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:16.159044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:16.159510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:16.159958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:16.161369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:16.178902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:16.178902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 143/292] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:15 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:16.178902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:16.223022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:16.247487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 150/293] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:15 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/users/5f1071b5409143c3acbbbef219c47539/roles/ca074f0bc20f4f9aaee65f81d2bcd83d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:27:16.291306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba604-ee56-443e-bf70-66b932facee7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:16.330049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:16.330759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:16.331050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:16.331331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:16.332191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=591f69135f614f759aed20e203122cc4) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:16.332513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:16.334139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9c9de-4ec2-4dfe-8995-a0c61d0bd08a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:16.342147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 144/294] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:16 2022] GET /identity/v3/role_assignments?user.id=591f69135f614f759aed20e203122cc4&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=a0903d733c974f51970116b3948f5ff8 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:17.147107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66896-3161-48f5-ab81-1c0c2ee6702b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.147628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66896-3161-48f5-ab81-1c0c2ee6702b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.147703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66896-3161-48f5-ab81-1c0c2ee6702b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.149484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 151/295] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:17.157365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f543dc-1c48-4fb4-a830-dd2646e7c33f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.158314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f543dc-1c48-4fb4-a830-dd2646e7c33f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.158827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f543dc-1c48-4fb4-a830-dd2646e7c33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.179624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f1f543dc-1c48-4fb4-a830-dd2646e7c33f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:17.181265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f1f543dc-1c48-4fb4-a830-dd2646e7c33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:17.192168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9c61-4ebf-4fdd-944f-03c3ebcd19c7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.192467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9c61-4ebf-4fdd-944f-03c3ebcd19c7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.192726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9c61-4ebf-4fdd-944f-03c3ebcd19c7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.194445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 152/296] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:17.201425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.201709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.201988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.223499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:17.225933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:17.271604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 145/297] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:17.273343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d685d-7c0b-4028-99a7-5e7cc46cf259 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.273875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d685d-7c0b-4028-99a7-5e7cc46cf259 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.274344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d685d-7c0b-4028-99a7-5e7cc46cf259 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.276209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 146/298] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:17.284517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.285312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.285791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.306182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:17.307765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:17.308772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 153/299] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:17.309676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.359636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:17.413219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 147/300] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:17.414473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.460741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216412e-3318-44c8-ac93-f373f017a420 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:17.479360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:17.497542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:17.498311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.498685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.498984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.500140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:17.501515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:17.503358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 154/301] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:17.504043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.553338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78dab-4939-4c31-bdff-98094767df2f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:17.591985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:17.592778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.593057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.593326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.604657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:17.612502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:17.613415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 148/302] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.637923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c842e4-e7ef-4494-8df5-67309b14959f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:17.643652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:17.683324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:17.684232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.684513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.684787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.696134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:17.704828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:17.705744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:17.706677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:17.710930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 155/303] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:17.711702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.733174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad991c-4ce7-4e4c-9d8b-7be2ea07d45a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:17.765697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:17.772855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:17.773528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.773809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.774083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] PATH_INFO: `/v3/groups` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.774809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:17.776267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:17.825630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e48d8-6139-4544-851b-4a05775b10c6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:17.837732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 149/304] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Jul 21 10:27:17 2022] POST /identity/v3/groups => generated 229 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:17.840034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.866896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:17.867909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:17.868437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:17.868989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:17.871090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:17.871658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:17.873865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:17.884883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 156/305] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:17.891694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:17.981839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:18.003508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200add3-e2ce-409f-98f8-3c033296df0a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.032601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e76c-ddbe-4445-af16-ecb7283d1a77 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.043174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:18.058585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.059413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.059611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.060556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.060686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:18.060943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:18.062331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:18.075542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 150/306] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:18.080894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:18.083590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.084486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.084773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.085046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] PATH_INFO: `/v3/users/a5afb2ec7a604967ba3f9fd752a6c957` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.103273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] RBAC: Authorizing `identity:get_user(user_id=a5afb2ec7a604967ba3f9fd752a6c957)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:18.105318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:18.107463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 157/307] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:17 2022] GET /identity/v3/users/a5afb2ec7a604967ba3f9fd752a6c957 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:18.114935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:18.240331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915e584-b205-4ed1-b856-98dc9f12d974 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.245778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f8c3c-6435-4c4a-88a4-93aacc7a0b02 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.290421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.291268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.291644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.292055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.293155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, user.id=a5afb2ec7a604967ba3f9fd752a6c957, scope.system=all) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:18.293584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:18.294800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.295293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b8ef-ec97-4220-b1f2-3abb0449ecb4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:18.295714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.296001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.296281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] PATH_INFO: `/v3/users/b2635acf532a4461b3382fab26038760` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.297830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] RBAC: Authorizing `identity:get_user(user_id=b2635acf532a4461b3382fab26038760)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:18.300120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:18.302232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 151/309] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:18 2022] GET /identity/v3/users/b2635acf532a4461b3382fab26038760 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:18.302647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 158/309] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 21 10:27:18 2022] GET /identity/v3/role_assignments?user.id=a5afb2ec7a604967ba3f9fd752a6c957&scope.system=all&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:18.308829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:18.340170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:18.440615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087774-ec38-4ec5-9b70-2e42094f78dd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.480925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.481809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.482087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.482366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] PATH_INFO: `/v3/system/users/b2635acf532a4461b3382fab26038760/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.483971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b2635acf532a4461b3382fab26038760, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:18.485368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757e0ef-2c58-4a66-ac76-aa82adfc3c2a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:18.564342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 152/310] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Jul 21 10:27:18 2022] PUT /identity/v3/system/users/b2635acf532a4461b3382fab26038760/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:18.661582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2a61-6ffe-4ddf-8308-29515703c5d4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.661869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2a61-6ffe-4ddf-8308-29515703c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.662154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2a61-6ffe-4ddf-8308-29515703c5d4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.663958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 159/311] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:18.670726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.671015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.671297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.690462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:18.691762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:18.778632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 153/312] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:18.796797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2a61-6ffe-4ddf-8308-29515703c5d4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:18.832424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:18.913311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2a61-6ffe-4ddf-8308-29515703c5d4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:18.955402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:18.956202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:18.956483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:18.956756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:18.968791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:18.976689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:18.977580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:18.978369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:18.982676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 160/313] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:18.985406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:18.998925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:19.096674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0835c-739d-4176-9e18-caf5f21aaeaf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:19.137079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:19.137763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:19.138046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:19.138325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:19.139116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:19.139534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:19.141000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:19.150031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 154/314] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:19.156430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:19.256518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:19.266825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51df30-ef3e-4e7a-953d-9160d6def8cb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:19.302820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:19.303750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:19.304034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:19.304303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] PATH_INFO: `/v3/users/5f1071b5409143c3acbbbef219c47539` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:19.305529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1071b5409143c3acbbbef219c47539)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:19.307019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:19.308939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 161/315] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:19 2022] GET /identity/v3/users/5f1071b5409143c3acbbbef219c47539 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:19.314966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:19.403599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:19.429427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943169f9-7908-44c0-9cf0-d0648a16bf9c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:19.466198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:19.466892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:19.467218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:19.467502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:19.538708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:19.540223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:19.623104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:19.653856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 155/316] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:19.655328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:19.763540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c869f4-e9dc-471e-a5b3-e9cfebbda199 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:19.800287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:19.801581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:19.802070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:19.802582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:19.804163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:19.805877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b91b5-58cf-4698-baa1-90f16c9c08d4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:19.813882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 162/317] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:19 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:19.815669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:19.927294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e4d89-27c1-4c58-a444-b6ba31060e9f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:19.964519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:19.965540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:19.966044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:19.966516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:19.967618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92, user.id=5f1071b5409143c3acbbbef219c47539) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:19.968163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:19.969804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb117dd-a9b2-4503-b54e-ffc959d76db6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:19.982690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 156/318] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:19 2022] GET /identity/v3/role_assignments?user.id=5f1071b5409143c3acbbbef219c47539&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:20.099136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c49e6-b1c2-4db1-9fe4-347c9c0faa5e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.099773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c49e6-b1c2-4db1-9fe4-347c9c0faa5e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.100254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c49e6-b1c2-4db1-9fe4-347c9c0faa5e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.102301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 163/319] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:20.108395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.108682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.108961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.129889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:20.131133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:20.160554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:20.212427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 157/320] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:20.235974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c49e6-b1c2-4db1-9fe4-347c9c0faa5e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:20.272751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:20.357429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c49e6-b1c2-4db1-9fe4-347c9c0faa5e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:20.395053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:20.395850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.396137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.396381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.397559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:20.399064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308059c-8c0c-418e-a005-e4880bfd6d13 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:20.400952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 164/321] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:20.407309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:20.519767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854fd2e-e7a3-42b9-95bb-eeb6d9e038b0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:20.565529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:20.566317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.566632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.566911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] PATH_INFO: `/v3/groups` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.567703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:20.569153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:20.576144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20675a1-ced7-492a-9d10-e0d03668e43f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.576650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20675a1-ced7-492a-9d10-e0d03668e43f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.577038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20675a1-ced7-492a-9d10-e0d03668e43f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.578511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 165/322] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:20.584077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e75-88ec-459c-ac33-f8fb80fe6f22 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.584371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e75-88ec-459c-ac33-f8fb80fe6f22 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.584651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42062e75-88ec-459c-ac33-f8fb80fe6f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.601441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-42062e75-88ec-459c-ac33-f8fb80fe6f22 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:20.602663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-42062e75-88ec-459c-ac33-f8fb80fe6f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:20.666892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 158/323] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Jul 21 10:27:20 2022] POST /identity/v3/groups => generated 226 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:20.680564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 166/324] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:20.704484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:20.734738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:20.821359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bca6d-5982-4ecc-8018-1eae943863f9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.821649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bca6d-5982-4ecc-8018-1eae943863f9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.821904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bca6d-5982-4ecc-8018-1eae943863f9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.822022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c197fb-e1be-46ea-a38e-a3474a67a409 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:20.824084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 167/325] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:20.829699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.829984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.830264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.852116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:20.853447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:20.869302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:20.870111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:20.870984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:20.871238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:20.877300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:20.883956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:20.892282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:20.893414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:20.894734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 159/326] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:20.899968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:20.941332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 168/327] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:20.965926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.033375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:21.058338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477673d-9518-499c-afe7-c4a76a61c43b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.120294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9d9b-2d93-4a4c-9533-043c17a691bd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.120880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.121571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.123198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.123198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.123386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:21.123768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.125824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.134972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 160/328] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:21.141315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.164839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.164839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.164839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.164839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.184777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:21.192985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:21.194024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:21.195517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 169/329] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:21.204577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.227420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:21.273889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdd41a-c3b0-4094-b11b-f46f0decab25 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.323823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.324691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.324975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.325254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] PATH_INFO: `/v3/users/a5afb2ec7a604967ba3f9fd752a6c957` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.329026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] RBAC: Authorizing `identity:get_user(user_id=a5afb2ec7a604967ba3f9fd752a6c957)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.330529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.332318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 161/330] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:21 2022] GET /identity/v3/users/a5afb2ec7a604967ba3f9fd752a6c957 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:21.339745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.359861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a645119-4175-408f-9287-fb205451fe09 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.362304 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:21.405813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.406768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.407349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.407624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.408396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:21.408702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.410159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.422362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 170/331] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:21.433791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.485666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ee10-faf4-410d-80d4-d50baf410490 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.528749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.530134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.530325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.530621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] PATH_INFO: `/v3/system/users/a5afb2ec7a604967ba3f9fd752a6c957/roles/ca074f0bc20f4f9aaee65f81d2bcd83d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.539391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a5afb2ec7a604967ba3f9fd752a6c957, role_id=ca074f0bc20f4f9aaee65f81d2bcd83d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.543321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.560435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:21.606804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a87e4-4a13-4990-8370-7af03edb7ca9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.638622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 162/332] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Jul 21 10:27:21 2022] PUT /identity/v3/system/users/a5afb2ec7a604967ba3f9fd752a6c957/roles/ca074f0bc20f4f9aaee65f81d2bcd83d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:21.647656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.649768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.650627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.651377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] PATH_INFO: `/v3/users/b2635acf532a4461b3382fab26038760` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.653625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] RBAC: Authorizing `identity:get_user(user_id=b2635acf532a4461b3382fab26038760)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.655417 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75df32-c974-46d7-8d6c-b86d065aad4f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.657878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 171/333] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:21 2022] GET /identity/v3/users/b2635acf532a4461b3382fab26038760 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:21.664690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:21.826940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:21.860849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf755c9-5295-4786-99b4-ecebe0013c78 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:21.903693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:21.907357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:21.907357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:21.907357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:21.907357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] RBAC: Adding query filter params (role.id=a0903d733c974f51970116b3948f5ff8, user.id=b2635acf532a4461b3382fab26038760, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:21.907357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:21.908037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f3d48-6a24-4141-b1f4-e8875c55e5ad None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:21.925355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 163/334] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 21 10:27:21 2022] GET /identity/v3/role_assignments?user.id=b2635acf532a4461b3382fab26038760&scope.system=all&role.id=a0903d733c974f51970116b3948f5ff8 => generated 510 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:23.526972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36d68-e5c2-40e9-9da2-9c893380a39c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:23.527990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36d68-e5c2-40e9-9da2-9c893380a39c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:23.528493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36d68-e5c2-40e9-9da2-9c893380a39c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:23.530478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 172/335] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:23.537072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71a3e-36f8-495c-9257-cc5e2fcd52c3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:23.538001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71a3e-36f8-495c-9257-cc5e2fcd52c3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:23.538518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71a3e-36f8-495c-9257-cc5e2fcd52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:23.561320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f8c71a3e-36f8-495c-9257-cc5e2fcd52c3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:23.563002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f8c71a3e-36f8-495c-9257-cc5e2fcd52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:23.655246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 164/336] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:23.692769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36d68-e5c2-40e9-9da2-9c893380a39c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:23.788212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4edaf-00a8-4d0c-b182-5a7cb702ee9e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:23.788818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4edaf-00a8-4d0c-b182-5a7cb702ee9e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:23.789304 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4edaf-00a8-4d0c-b182-5a7cb702ee9e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:23.792058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 165/337] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:23.798730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:23.799364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:23.799838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:23.840049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:23.842295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:23.856843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36d68-e5c2-40e9-9da2-9c893380a39c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:23.926602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:23.927917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:23.928406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:23.928930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:23.946925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:23.955289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:23.961875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 166/338] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:23.966495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:23.967800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:23.968630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 173/339] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:23.975790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:24.021536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:24.029280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:24.201911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e8ebf-160a-4948-8fdf-4e357893a6d4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:24.222208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfec19-a5b5-4df4-a58b-b7be36ae4ff4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:24.296671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:24.297405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:24.297695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:24.297998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:24.298783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:24.299094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:24.300510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:24.316900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:24.318208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:24.318716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:24.319183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:24.337017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 167/340] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:24.358647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:24.361970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:24.371286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:24.372468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:24.373930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 174/341] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:24.396413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:24.477253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:24.536575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:24.559516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45924ff3-3290-420d-b95e-c2e72262f956 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:24.607931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dd6a8-0254-462e-afcc-03147efdf1a5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:24.624846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:24.625891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:24.626378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:24.626896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:24.679803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:24.680539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:24.680823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:24.681099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:24.681863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:24.682168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:24.683904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:24.701589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 175/342] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:24.707299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:24.815809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:24.817648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b879ff-e026-4e1d-a2fa-ea8d2863ee8c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:24.902580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2f2b8-70e0-4769-b29b-405f88239ee5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:24.946855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:24.969375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:24.970112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:24.970435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:24.970955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:24.974980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.097324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 168/343] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:25.098521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebf1cb-7f51-4a40-9e7d-e299a8e1732f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.098839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebf1cb-7f51-4a40-9e7d-e299a8e1732f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.099116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebf1cb-7f51-4a40-9e7d-e299a8e1732f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.099816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:25.100589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 169/344] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:25.101290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:25.108171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.108571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.108862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.143943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:25.145288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:25.262011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 176/345] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:25.263084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.272375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 170/346] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:25.273279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.296940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.399813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c0b4-e243-42f5-a04b-efccaabc6793 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:25.409587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.424964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93597-baa1-4dee-a33e-27adf83ddb83 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:25.441157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:25.442037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.442319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.442626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.443842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:25.445301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb60b4-51d1-4d89-9d6f-7aa0ba02424a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:25.447052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 177/347] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:25.448818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a85a-5e78-4201-8428-9bd18d2e8b09 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.449338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a85a-5e78-4201-8428-9bd18d2e8b09 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.449618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a85a-5e78-4201-8428-9bd18d2e8b09 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.455430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 178/348] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:25.456290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a85a-5e78-4201-8428-9bd18d2e8b09 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.484960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:25.485858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.486250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.486536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.487839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:25.489526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c0691-e525-409b-84c3-f12ae5ddf6d4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:25.491852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 171/349] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:25.493282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.493673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.493957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.519206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:25.520713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:25.580403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.594294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a85a-5e78-4201-8428-9bd18d2e8b09 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:25.621696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 172/350] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:25.628155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.650437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:25.652082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.652517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.652958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.668419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:25.688571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:25.689543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:25.690343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:25.691507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 179/351] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:25.693382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.726113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.746369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2144d-8af4-4266-ab38-3fbbf3d83e7d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:25.795122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:25.796720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.796720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.796720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.804177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:25.805639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dba5a-bc2e-479a-845f-beb4ce28fe80 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:25.808292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 173/352] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:25.810615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e2004-34a6-4df8-ad93-d595eab9633e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.810615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e2004-34a6-4df8-ad93-d595eab9633e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.810615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e2004-34a6-4df8-ad93-d595eab9633e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.812426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 174/353] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:25.813170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e2004-34a6-4df8-ad93-d595eab9633e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.852974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f1b3-7a56-4242-9d59-548a0a5115cf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:25.900571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:25.901427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:25.901784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:25.902293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:25.903271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:25.904737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:25.906745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 180/354] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:25.907882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:25.947011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:25.973134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e2004-34a6-4df8-ad93-d595eab9633e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.017222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.018031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.018316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.018621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.034439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:26.043464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:26.044511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:26.045303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 175/355] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:26.045957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.047040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39094302-ebc1-443e-ad8e-659785f20f17 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.062292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.095067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.095753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.096033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.096310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.097996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:26.098311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:26.099742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b2cc-b3fd-48fa-87c9-87c2e900b3dc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:26.110728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 181/356] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:26.112397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c61c92-503c-4282-9b4b-d56c4e193cd4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.112694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c61c92-503c-4282-9b4b-d56c4e193cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.112977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c61c92-503c-4282-9b4b-d56c4e193cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.134022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-56c61c92-503c-4282-9b4b-d56c4e193cd4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:26.136197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-56c61c92-503c-4282-9b4b-d56c4e193cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:26.185723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2197-0347-438f-94b3-f54fe7c5fdde None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.221240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 182/357] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:26.221618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc1a0-0c74-468c-95a8-5b1868fc92b4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.221859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc1a0-0c74-468c-95a8-5b1868fc92b4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.222130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc1a0-0c74-468c-95a8-5b1868fc92b4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.224403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 183/358] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:26.224946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bc1a0-0c74-468c-95a8-5b1868fc92b4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.238502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.239461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.239752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.240017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.240861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=fc9b385b87ca4f5aab27028f16841078) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:26.241173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:26.243330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:26.247981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.250333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 176/359] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=a0903d733c974f51970116b3948f5ff8 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:26.251657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.376406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.376810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bc1a0-0c74-468c-95a8-5b1868fc92b4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.416066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c50d5-3cd2-471d-a741-af36a9e9000a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.428677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.429612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.430214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.430327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.431158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=fc9b385b87ca4f5aab27028f16841078) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:26.431617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:26.435287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:26.443387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 184/360] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:26.444300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.471719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.472414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.472692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.472966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.473734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:26.474041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:26.475498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c61fc8-b183-40a6-8a7c-31ca0cbca99d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:26.488723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 177/361] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:26.490425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.490737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.491052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.523047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:26.524306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:26.562569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.596121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784e6f3-f57d-4564-9895-ddb81829c742 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.607248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 178/362] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:26.608253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.663756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.664560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.664850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.665133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.712598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.752596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:26.754171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:26.762265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022242ee-55ed-49e2-ba59-376e2c49a515 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:26.826337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:26.827249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:26.827534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:26.827813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:26.843676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:26.853111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:26.854014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:26.856011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 179/363] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:26.857184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:26.878316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:26.908907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 185/364] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:26.909768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.005639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9829-39f9-44fc-b841-608297a1eab3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.041009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.052509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.053625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.056838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.056838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.057058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ea8de-72bb-4f77-a1eb-f895aff6412e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.103619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.105086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.105373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.105825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.121506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:27.132032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:27.132991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:27.133802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:27.134423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 186/365] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:27.135315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.162250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.162250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.194795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.278138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c660c7-7e1b-40dc-b167-ca514362cdd5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.307215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 180/366] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:27.308499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.325774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.326525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.326837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.327179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.328460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:27.328783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.330614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.343243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 187/367] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.344419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.350739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.456390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd083c-67d5-4544-81dd-2622a325155e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.489178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.496845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f708365-ec47-4f40-bd28-0cb1d3710dcd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.510052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.511880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.512164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.512434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.514018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.519871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.521858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 181/368] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.523379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.556890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.558018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.558398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.558737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.559510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:27.559816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.561348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.577591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 188/369] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.578602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.646819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.655870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b9736-c86a-446a-9ca6-3cd444a23fc5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.708075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.708956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.709236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.709509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.711066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.712658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.715157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 182/370] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.715823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95829c09-141b-45c0-b678-3134aff017f7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.716375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.756688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.757560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.757839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.758108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] PATH_INFO: `/v3/users/a5afb2ec7a604967ba3f9fd752a6c957` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.759576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] RBAC: Authorizing `identity:get_user(user_id=a5afb2ec7a604967ba3f9fd752a6c957)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.761110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.762937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 189/371] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/users/a5afb2ec7a604967ba3f9fd752a6c957 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.763775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.801731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:27.832609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16057-c261-41b0-b107-d80f097d0473 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.871322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfe5e6-7488-4e3f-8528-091148226986 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:27.874481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.875397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.875675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.875949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.877158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:27.878614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:27.880553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 183/372] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:27.881334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:27.911637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:27.912391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:27.912676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:27.912944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:27.918701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.016414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.018339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.020744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9484-78b7-48b7-a195-77023aafac53 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.074639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.092266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.093542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.093998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.094448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.097817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.100476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.104110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 184/373] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:28.105605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.202127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 190/374] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:28.207646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.291254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.297822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d9617-6a68-4f70-bf74-45552f7e4897 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.362914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.363863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.364440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.364887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.366617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] RBAC: Adding query filter params (role.id=ca074f0bc20f4f9aaee65f81d2bcd83d, user.id=a5afb2ec7a604967ba3f9fd752a6c957, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:28.367077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.370640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.393545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eaa26-fbb2-445f-b143-d5ac0fa845c7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.396806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 185/375] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/role_assignments?user.id=a5afb2ec7a604967ba3f9fd752a6c957&scope.system=all&role.id=ca074f0bc20f4f9aaee65f81d2bcd83d => generated 510 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:28.399235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.479105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.503112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.503792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.504074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.504355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.505209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:28.505595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.507146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.521000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 191/376] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 260 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:28.522131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.564006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbec21-c6e8-44ba-b631-7d68f31e8cac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.612646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875566-567a-4575-b218-7354364a694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.613383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875566-567a-4575-b218-7354364a694a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.613604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875566-567a-4575-b218-7354364a694a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.613882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875566-567a-4575-b218-7354364a694a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.614766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875566-567a-4575-b218-7354364a694a None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:28.615079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875566-567a-4575-b218-7354364a694a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.616562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875566-567a-4575-b218-7354364a694a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.633278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 186/377] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=a0903d733c974f51970116b3948f5ff8 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:28.695406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4928d3-a021-409d-b977-9c4323bbae90 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.718080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f934f4-2797-4c2c-a1bb-4b5090846360 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.718612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f934f4-2797-4c2c-a1bb-4b5090846360 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.718688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f934f4-2797-4c2c-a1bb-4b5090846360 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.720076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 187/378] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:28.733515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.733824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.734071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.744899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.746881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.747430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.747704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] PATH_INFO: `/v3/projects/549a3510fe3f48278464436ac5f2a88c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.749151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] RBAC: Authorizing `identity:get_project(project_id=549a3510fe3f48278464436ac5f2a88c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.751144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.754525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 192/379] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/projects/549a3510fe3f48278464436ac5f2a88c => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:28.764537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.774864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:28.776186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:28.821189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.867095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:28.936969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e0836-ac7c-4779-8fc9-c2515be330c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:28.938901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 188/380] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:28.977240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:28.979194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:28.980152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:28.980619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:28.980889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] PATH_INFO: `/v3/groups/9a0e181395b143bb9262dadc1f036667` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:28.991457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] RBAC: Authorizing `identity:get_group(group_id=9a0e181395b143bb9262dadc1f036667)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:28.992963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:28.996406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 193/381] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/groups/9a0e181395b143bb9262dadc1f036667 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:29.002332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.108648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccfdae-dfa4-4c02-849b-0db6d31df093 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.152490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.153639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.154110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.154604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.170570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:29.178659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:29.179596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:29.180382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 189/382] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:29.189733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.204041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:29.228520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c25dfb-cc3d-4b40-a8b6-649e17123d18 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.280900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:29.293739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.294781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.302721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.302721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.302721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] RBAC: Adding query filter params (group.id=9a0e181395b143bb9262dadc1f036667, role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=549a3510fe3f48278464436ac5f2a88c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:29.302721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:29.303612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:29.310311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e63ba-340d-40b4-9b54-cf65eebbcbbd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.310928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 194/383] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/role_assignments?group.id=9a0e181395b143bb9262dadc1f036667&scope.project.id=549a3510fe3f48278464436ac5f2a88c&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 260 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:29.352459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.353132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.353409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.353681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.355211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:29.355689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:29.357386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a13036-2745-448b-a2e1-78e69455f93e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:29.367638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 190/384] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:29.374412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.463472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99e09-ef8a-4a2d-b934-354e8069296f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.464749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99e09-ef8a-4a2d-b934-354e8069296f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.465230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99e09-ef8a-4a2d-b934-354e8069296f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.467505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 191/385] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:29.483241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.483886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.484366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.512725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544417b9-91ea-47c6-8ae8-400bd680c33a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.518335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:29.520021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:29.562586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.562586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.562586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.562944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.563918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:29.565381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:29.568597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 195/386] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:29.576032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.628713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 192/387] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:29.665196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.669044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:29.715024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:29.797360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bad57-3b1d-4c78-897d-ef75e79785d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.832933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557e6ab-faa6-4fcf-8eef-61ce8ed8973e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:29.840493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.841677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.842170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.842664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.855422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:29.864889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:29.865821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:29.866706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:29.869537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 193/388] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:29.880102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.924372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:29.925263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:29.925545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:29.925817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:29.931303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:29.933831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:29.935678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 196/389] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:29.941791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:29.955734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:30.042456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c33d2-5e76-4ffb-8540-81ecc1511646 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:30.096565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:30.097618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:30.098109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:30.098595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:30.099668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:30.100170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:30.101886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:30.112879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 194/390] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:30.122934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:30.172204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a62908-4d48-4377-bb2c-3aafb164f453 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:30.224422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:30.241861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:30.242785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:30.243069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:30.243343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:30.246979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, group_id=f48a4139dc8540d5aba9a23defbd1e77, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:30.248778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:30.298010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b9fbe-f4b8-4460-b7bb-0fb963a0aa00 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:30.329952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:30.346749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 197/391] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 21 10:27:29 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:27:30.351218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:30.352177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:30.352459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:30.352734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:30.354116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:30.356493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:30.358729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 195/392] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:30 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:30.364423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:30.493401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4ebb0-9ef3-44ee-abc3-8b4bc23fa5fd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:30.544323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:30.545207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:30.545484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:30.545754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:30.546984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:30.548543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08496c-622e-4437-b6d5-15a688c0324d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:30.550576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 198/393] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:30 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:30.556336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:30.688040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c32f-78df-4794-922a-c9dfc15a7804 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:30.736903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:30.737799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:30.738075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:30.738341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:30.744704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b385b87ca4f5aab27028f16841078, group_id=f48a4139dc8540d5aba9a23defbd1e77, role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:30.746697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39944157-82b3-4bca-bd85-6486ed32d4a1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:30.821327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 196/394] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 21 10:27:30 2022] PUT /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:31.196387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeec0de-fc51-4a49-a080-f35a8ad4daa4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.196987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeec0de-fc51-4a49-a080-f35a8ad4daa4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.196987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeec0de-fc51-4a49-a080-f35a8ad4daa4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.199569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 199/395] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:31.205349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685f34a-364f-466a-9bb6-f4c9359678e8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.205642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685f34a-364f-466a-9bb6-f4c9359678e8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.205919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685f34a-364f-466a-9bb6-f4c9359678e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.225136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a685f34a-364f-466a-9bb6-f4c9359678e8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:31.226311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a685f34a-364f-466a-9bb6-f4c9359678e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:31.285589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:31.314835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 197/396] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:31.347065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeec0de-fc51-4a49-a080-f35a8ad4daa4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:31.419859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:31.462151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8fe3b-5ca5-4ea8-8e95-f94b7fd0b470 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.462531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8fe3b-5ca5-4ea8-8e95-f94b7fd0b470 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.462749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8fe3b-5ca5-4ea8-8e95-f94b7fd0b470 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.464685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 198/397] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:31.470927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.471208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.471486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.478528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeec0de-fc51-4a49-a080-f35a8ad4daa4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:31.502409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:31.504280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:31.549974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:31.551157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.553509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.554144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.576734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:31.593545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:31.594508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:31.595340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:31.595968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 200/398] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:31.600456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:31.610735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 199/399] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:31.643269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:31.747444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16810189-c9b8-4b5a-91f6-64b17d6fbcf6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:31.770929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:31.785838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:31.786677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.786961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.787238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.787991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:31.788457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:31.790207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:31.800311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 201/400] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:31.805732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:31.916764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b032b-2998-44c1-9230-6a1f07ca69e8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:31.939796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23234ec-20e0-4153-8b55-bc080c2ae869 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:31.972444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:31.972929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:31.973889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:31.974291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:31.974860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:31.976373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:31.978093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:31.981100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 202/401] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:31 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:31.992078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.022470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.023208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.023485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.023753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.041168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:32.050910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:32.051882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:32.052763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 200/402] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:32.063832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.103129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:32.226184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bb2-a2bd-4971-8d4a-e063b06a4b00 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.279249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ce3a-109c-4913-9d8d-952f04ba9b07 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.311736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.313642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.314149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.314670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.317240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:32.320624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87cee0-809d-458d-89e2-203673be52ea None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:32.330582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 203/403] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:31 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:32.330582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a84bc-192a-41ae-81a9-db083d2a70b0 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.330582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a84bc-192a-41ae-81a9-db083d2a70b0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.330582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a84bc-192a-41ae-81a9-db083d2a70b0 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.331232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 204/404] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:32.340860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a84bc-192a-41ae-81a9-db083d2a70b0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.370426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.372298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.373007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.373725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.375378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:32.376198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:32.378947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0aeb2-802a-4bb6-b0fa-3c7cd0c2dc36 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:32.390823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 201/405] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:32.392522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.393036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.393642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.413219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:32.414892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:32.445788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:32.494239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 202/406] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:32.495417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.512459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a84bc-192a-41ae-81a9-db083d2a70b0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.542154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:32.567042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.568316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.568316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.568536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.571010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, group_id=f48a4139dc8540d5aba9a23defbd1e77, role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:32.572801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:32.606344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deaf62-4f37-4219-ac3a-f3689f94dc6c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.643291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.644431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.644861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.645312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.646690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:32.648292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:32.650516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 203/407] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:32 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:32.652083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.689401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:32.689770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 205/408] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 21 10:27:32 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:32.691127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.766622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621a0ec-34ae-4637-98e7-4fe3446fa431 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.805131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.806077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.806203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.806401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.819383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:32.828176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:32.829798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:32.836452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 204/409] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:32.847672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.850991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecd5a7-74d5-4667-b318-cc3b817808d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:32.906739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:32.908032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:32.908513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:32.908977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:32.910806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:32.912630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:32.914804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 206/410] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:32 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:32.921711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:32.969984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:32.990329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:33.000041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e160f4-5218-4553-b213-280db8e7dbd1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.040180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.040798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514ee6d-a291-4661-9cbb-20ad43a94671 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.040919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.041141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.041524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.042778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:33.043088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.044670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.053835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 205/411] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:33.060425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.112424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.113334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.113663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.113935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.119430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78ce352aee4c6b88c8521cf8468a92, group_id=f48a4139dc8540d5aba9a23defbd1e77, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.121653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b9e9b-3f2a-433e-995c-2f2a661f91a5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.180996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 207/412] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 21 10:27:32 2022] PUT /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92/groups/f48a4139dc8540d5aba9a23defbd1e77/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:27:33.184788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da94b-3754-4120-b2ba-e361b1d12cdd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.185075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da94b-3754-4120-b2ba-e361b1d12cdd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.185351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da94b-3754-4120-b2ba-e361b1d12cdd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.186931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 208/413] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:33.199420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.199782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.200061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.217645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4199a9-b551-4b50-aef9-8df9d17dabad None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.259587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.259803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:33.260528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.260999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.261231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:33.261325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] PATH_INFO: `/v3/groups/9a0e181395b143bb9262dadc1f036667` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.262674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] RBAC: Authorizing `identity:get_group(group_id=9a0e181395b143bb9262dadc1f036667)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.264294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.270705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 206/414] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/groups/9a0e181395b143bb9262dadc1f036667 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:33.272446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.306961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:33.388531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 209/415] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:33.413452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.428069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58fbd-eec2-4473-9de9-810118b0a908 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.454926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:33.480466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.481614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.482078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.482513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] PATH_INFO: `/v3/projects/549a3510fe3f48278464436ac5f2a88c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.483971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] RBAC: Authorizing `identity:get_project(project_id=549a3510fe3f48278464436ac5f2a88c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.485820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.488215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 207/416] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/projects/549a3510fe3f48278464436ac5f2a88c => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:33.496283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.544119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee04a20-d260-4009-a046-0e3b82dd587e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.581814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.582633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.582917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.583196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.596830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:33.606068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:33.607410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:33.608650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 210/417] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:33.616981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.684493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:33.697335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff1716-8942-4e19-8688-9792189529c5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.741998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:33.749033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.750215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.753079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.753079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] PATH_INFO: `/v3/projects/549a3510fe3f48278464436ac5f2a88c/groups/9a0e181395b143bb9262dadc1f036667/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.753274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3fc1c-3072-4660-ba93-8589afd7c3ea None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.753848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] RBAC: Authorizing `identity:create_grant(project_id=549a3510fe3f48278464436ac5f2a88c, group_id=9a0e181395b143bb9262dadc1f036667, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.755946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641d3dc-004b-4a19-89fb-e2ab10f8350a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.792282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.792994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.793273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.793543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.794295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:33.794619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:33.796071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb27-c1b7-4bd0-8311-c2b14b2570b8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:33.809716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 211/418] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:33.809716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caf0ae-a00c-44ad-a10f-c500ec2ff62e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.809716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caf0ae-a00c-44ad-a10f-c500ec2ff62e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.809716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caf0ae-a00c-44ad-a10f-c500ec2ff62e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.810272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 212/419] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:33.812898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3caf0ae-a00c-44ad-a10f-c500ec2ff62e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:33.833596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 208/420] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 21 10:27:33 2022] PUT /identity/v3/projects/549a3510fe3f48278464436ac5f2a88c/groups/9a0e181395b143bb9262dadc1f036667/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:33.835575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.838337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.838337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.858750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:33.860403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:33.937375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3caf0ae-a00c-44ad-a10f-c500ec2ff62e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:33.945575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 209/421] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:33.981654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:33.982340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:33.982653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:33.982921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:33.988089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.008010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.062955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.064162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:34.155986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.183563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5b016-d041-43e7-985a-c53692568d3c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.209587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 213/422] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:34.219187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.238327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.239588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.240031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.240497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.255462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:34.266062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:34.267627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:34.268428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:34.270753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 210/423] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:34.279045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.300415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.404694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee633fd-638c-4df1-b7e6-986e315c6ef6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.434836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aaec4-2756-4c76-9e6e-6b57d7d343db None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.482982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.483579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.483915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.484193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.484939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:34.485240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.486655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:34.500967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.501818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.502095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.502363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.507830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.507830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:34.507830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 214/424] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:34 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:34.513320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.515015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 211/425] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:34.521915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.529597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.639523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ffc1d-deee-492e-a3a6-34cbba4c5e2a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.677508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.724918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.726295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.726817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.727263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.729204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.733117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:34.741519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019f223-1de7-4cc9-9914-0af903d55f28 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.749278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 215/426] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:34 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:34.763712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:34.820973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.821620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.821900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.822171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.952006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74c6a-4388-47af-8cdc-23fb932b1683 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:34.989786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.991262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:34.991407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:34.991921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:34.992198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:34.992469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:34.993422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:34.993526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=fc9b385b87ca4f5aab27028f16841078) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:34.993765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:34.995287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:35.003187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 216/427] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:34 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=a0903d733c974f51970116b3948f5ff8 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:35.130147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 212/428] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:35.136298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:35.209304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:35.266436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435000f-8872-4f36-89fe-2540d69e58f0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:35.307123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:35.309145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.309425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.309695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] PATH_INFO: `/v3/projects/fc9b385b87ca4f5aab27028f16841078` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.310949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=fc9b385b87ca4f5aab27028f16841078)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:35.312471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9d91e-b7e9-4a82-ab24-b9cccba0e2e6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:35.314695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 217/429] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:35 2022] GET /identity/v3/projects/fc9b385b87ca4f5aab27028f16841078 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:35.321011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:35.435313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfba62-84e8-45d4-90a7-979518a1d907 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.435624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfba62-84e8-45d4-90a7-979518a1d907 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.435921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfba62-84e8-45d4-90a7-979518a1d907 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.437680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 218/430] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:35.444780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.445448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.445925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.469943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b45fce-2bc0-4cb4-a022-fc7154a32192 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:35.471663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:35.473093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:35.474802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:35.559103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:35.559932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.560211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.560489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.564346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:35.565821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:35.567799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 213/431] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:35 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:35.577895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:35.587644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 219/432] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:35.613869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:35.683128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:35.724243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32be5-16de-40e0-9b3e-ed9f7c416cea None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:35.750198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968262d-713d-4822-88f9-fafc99ea86a0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:35.772982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:35.773648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.773932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.774209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.775173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=fc9b385b87ca4f5aab27028f16841078) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:35.775495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:35.777186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575c1ea-f40d-4d41-aa56-f272c6037118 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:35.785201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 214/433] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:35 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=fc9b385b87ca4f5aab27028f16841078&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:35.804560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:35.805421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.805698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.805973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.823598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4725-22ff-4e28-9f80-b118994a9fe7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.823825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4725-22ff-4e28-9f80-b118994a9fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.824076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4725-22ff-4e28-9f80-b118994a9fe7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.825468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 215/434] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:35.830597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:35.832485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:35.832772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:35.833045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:35.836709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:35.837720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:35.838727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 220/435] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:35.844634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:35.865800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:35.867332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:35.913500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:35.973646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 216/436] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:36.002950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.007482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90f9f7-6825-412a-86ee-b50b23538c11 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.023702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.053967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.054698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.055045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.055318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.056219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:36.056590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.058929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.069601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 221/437] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:36.076064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.151006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaddf17-4016-4428-a4b7-a4786566a403 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.207216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.208034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.208306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.208592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.221166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:36.225905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa08300-1c1c-472c-b499-36d4afafee47 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:36.229005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:36.230537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 10:27:36.231478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:36.234251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 217/438] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:36.243369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.293971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.297517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.298372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.298803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.299198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.365251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.400187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.401787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.404662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc82e97-14cd-4214-8264-d71461919907 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.444055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.444788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.445085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.445361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.446115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:36.446413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.449777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.459566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 218/439] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:36.465566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.522475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 222/440] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:36.530036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.575577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.592820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6911-65fd-4806-865f-bb46eb3b23b1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.616362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.633881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.634565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.634865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.635141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.650109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40487a5-2ac4-44f6-ae8d-b8e0f007303f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.696527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.697929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.698375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.698880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.700480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.702048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27cc82-178c-49d3-af9d-70a03e869738 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.703875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 223/441] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:36.704966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf899-8a59-4f2c-a58c-2866155e404f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.705251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf899-8a59-4f2c-a58c-2866155e404f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.705524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf899-8a59-4f2c-a58c-2866155e404f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.706819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 224/442] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:36.711775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcf899-8a59-4f2c-a58c-2866155e404f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.718915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.720382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e43e55-d189-453f-b8d8-d44d582f6437 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.818530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcf899-8a59-4f2c-a58c-2866155e404f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:36.822661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.851052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 219/443] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:36.852537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.852829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.853107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.855878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:36.856828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:36.857098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:36.857495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:36.858736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:36.860191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:36.862018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 225/444] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:36.862993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.875607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:36.875785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:36.876751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:36.946564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 220/445] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:36.947549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:36.974228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27b217-5150-4326-a5fe-cee3884b9144 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.009749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.010861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.011107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.011415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] PATH_INFO: `/v3/projects/5b78ce352aee4c6b88c8521cf8468a92` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.012482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78ce352aee4c6b88c8521cf8468a92)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.013944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.015658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 226/446] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/projects/5b78ce352aee4c6b88c8521cf8468a92 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:37.016419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.052641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57778c0f-d92e-4414-8dfc-622ef24d7860 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.070498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.089549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.090202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.090528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.090988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.091818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:37.092133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.093634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.100566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 221/447] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:36 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:37.101555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.110641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.138969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836d8d3-6aed-4216-a333-f8f29423f173 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.185204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.186018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.186297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.186596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.205894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:37.214445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:37.215716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:37.216636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 227/448] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:37.222033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.242426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09fa30-2343-4b59-af7c-f45bffabf910 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.296241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.297100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.297378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.297649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] PATH_INFO: `/v3/groups/f48a4139dc8540d5aba9a23defbd1e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.298906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] RBAC: Authorizing `identity:get_group(group_id=f48a4139dc8540d5aba9a23defbd1e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.300346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.301973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 222/449] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/groups/f48a4139dc8540d5aba9a23defbd1e77 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:37.308519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.368155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.371241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12866e7-d738-4104-950c-512d0776fd14 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.408003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.408673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.408953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.409247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.409979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:37.410330 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.411792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.414744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.424067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 228/450] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:37.431663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.441019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f294-809a-4028-9aeb-2df9ccb6086b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.484855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.485980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.486524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.487053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.488681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] RBAC: Adding query filter params (group.id=f48a4139dc8540d5aba9a23defbd1e77, role.id=a0903d733c974f51970116b3948f5ff8, scope.project.id=5b78ce352aee4c6b88c8521cf8468a92) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:37.489237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.491980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.501170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 223/451] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/role_assignments?group.id=f48a4139dc8540d5aba9a23defbd1e77&scope.project.id=5b78ce352aee4c6b88c8521cf8468a92&role.id=a0903d733c974f51970116b3948f5ff8 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:37.563416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9bee2-a53b-4ac4-8952-1188ee391834 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.604029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.604746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.605040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.605298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.680303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.686472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.688612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.798951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 229/452] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 10:27:37.805452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:37.920369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94425542-f359-47ea-8fe9-a6e88313e896 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:37.935495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:37.957714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:37.958524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:37.958835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:37.959114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] PATH_INFO: `/v3/projects/549a3510fe3f48278464436ac5f2a88c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:37.960298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] RBAC: Authorizing `identity:get_project(project_id=549a3510fe3f48278464436ac5f2a88c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:37.961728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:37.963520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 224/453] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/projects/549a3510fe3f48278464436ac5f2a88c => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:37.969529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:38.069352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:38.077218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69116-e3b8-4030-831b-36deb83c968e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:38.113321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:38.114172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:38.114452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:38.114754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] PATH_INFO: `/v3/groups/9a0e181395b143bb9262dadc1f036667` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:38.115926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] RBAC: Authorizing `identity:get_group(group_id=9a0e181395b143bb9262dadc1f036667)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:38.117361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165809b-2d26-4ca5-af95-158e79dbd790 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:38.119019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 230/454] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 21 10:27:37 2022] GET /identity/v3/groups/9a0e181395b143bb9262dadc1f036667 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:38.125911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:38.231358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a810-0c11-4d73-a4b9-bfdf725756f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:38.268014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:38.268687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:38.268967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:38.269241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:38.270064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] RBAC: Adding query filter params (group.id=9a0e181395b143bb9262dadc1f036667, role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=549a3510fe3f48278464436ac5f2a88c) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:38.270373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:38.271832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d88d4-7930-4a0e-97f6-8d1ef34ce115 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:38.277746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 225/455] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 21 10:27:38 2022] GET /identity/v3/role_assignments?group.id=9a0e181395b143bb9262dadc1f036667&scope.project.id=549a3510fe3f48278464436ac5f2a88c&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:40.830166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5eb63-b452-4528-9e11-867f56331827 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:40.830753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5eb63-b452-4528-9e11-867f56331827 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:40.830753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5eb63-b452-4528-9e11-867f56331827 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:40.832397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 231/456] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:40.838992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:40.839493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:40.839716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:40.863227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:40.864419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:40.929679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:40.969735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 226/457] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:41.015573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5eb63-b452-4528-9e11-867f56331827 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:41.168287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5eb63-b452-4528-9e11-867f56331827 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:41.217772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:41.218604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.218893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.219170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.231504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:41.233216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:41.259329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:41.260846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:41.261704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:41.261704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:41.261704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:41.261704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:41.261936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 232/458] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:41.269974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:41.412139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314e8d-aa01-4462-86c3-69f313ea7952 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:41.449806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:41.450859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.451350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.451829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.452773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:41.453279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:41.454890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6ea7a-d624-4175-b117-110ed6f73994 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:41.472187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 227/459] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:41.477273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:41.507005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:41.537882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278141a-bf64-4f65-bf3b-021e67166dad None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.538660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278141a-bf64-4f65-bf3b-021e67166dad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.539134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278141a-bf64-4f65-bf3b-021e67166dad None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.540958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 228/460] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:27:41.546993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579f35-8c55-480d-bb77-9a957ed76f1f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.547571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579f35-8c55-480d-bb77-9a957ed76f1f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.548040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579f35-8c55-480d-bb77-9a957ed76f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.591650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-41579f35-8c55-480d-bb77-9a957ed76f1f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:41.593238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41579f35-8c55-480d-bb77-9a957ed76f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:41.597738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6eee5-3eed-4985-ac65-505114363ff8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:41.638491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:41.639523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.639810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.640090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.640822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:41.642321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9d5d8-a72d-4352-a29a-d34e98cef056 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:41.643068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:41.711472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 229/461] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:41.714278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d5de4-7cc0-4877-9008-93bcecdc9d13 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.714278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d5de4-7cc0-4877-9008-93bcecdc9d13 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.714647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d5de4-7cc0-4877-9008-93bcecdc9d13 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.716161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 230/462] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:41.717087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf0c7-782f-4d1a-b11c-6a6b6caff6e1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.717424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf0c7-782f-4d1a-b11c-6a6b6caff6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.717804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf0c7-782f-4d1a-b11c-6a6b6caff6e1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.719412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 231/463] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:41.725898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.726180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.730084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.731235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 233/464] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:41 2022] POST /identity/v3/users => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:41.733229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:41.733752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:41.734226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:41.759910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:41.761825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:41.765438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:41.767158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:41.846858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 232/465] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:41.847443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:41.853678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 234/466] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:41.875889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:41.882194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.014116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.019727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b562d0-a98f-4b8b-a750-1cf9f5a269ee None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.030696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4fcd0-a437-45ad-bed2-4efdb22c9a5e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.062978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.063886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.064169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.064445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.079399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.081216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.081373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.082017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.082295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.082585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.097574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.099076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:42.102021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:42.102915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:42.103772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:42.103772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:42.103772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.103772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:42.103772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 233/467] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:42.104767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:42.125340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:42.126261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:42.127129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:42.127129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:42.127129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.127129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:42.130913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 235/468] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:42.132212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.222866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.249898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c1984-9401-4c81-93b8-91465a2220ea None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.282091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37287-11d7-4480-a7d2-5e3e91dbd6c8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.301705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.303127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.303721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.304411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.321594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.323602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.327926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.328885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.329413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.329813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.330699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:42.331085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.332615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.334283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:42.347941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:42.349145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:42.350081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:42.350081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:42.350081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:42.350081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:42.352688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 234/469] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:42.353972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.354999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 236/470] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:42.363947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.473577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bda6b-4e5b-4bd5-ab6b-da9aa8bcc01b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.509591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.510837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.511137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.511421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.512147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:42.512454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.513835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.516762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1926-4f05-4abc-821e-f45871a3d201 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.526438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 235/471] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:42.527293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.564989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.584321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.584994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.585271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.585550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.586318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:42.586694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.588217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98762e6-9a60-4866-8623-4b808fe05c6f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.600341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 237/472] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:42.602765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86af90-b33a-4290-a734-9df90c1b1ef1 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.603191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86af90-b33a-4290-a734-9df90c1b1ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.603846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86af90-b33a-4290-a734-9df90c1b1ef1 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.611029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:42.611205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 238/473] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:42.611778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86af90-b33a-4290-a734-9df90c1b1ef1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.665207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef10a9-25f1-4aa8-a0e3-2ffca7b32a89 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.711468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.712183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.712477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.712749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.713541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.714990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.734474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86af90-b33a-4290-a734-9df90c1b1ef1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.768224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:42.768884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.769162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.769570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.770380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:42.771836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83be5b-5cc1-4cd0-9bf0-dd1230d1a4fc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:42.802952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 236/474] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:42 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:42.803757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:42.914567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 239/475] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:42 2022] POST /identity/v3/users => generated 249 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:42.914567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:42.914567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:42.914567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:42.946845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:42.948228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:42.970286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d4cbf-bc1e-4fe9-82d2-2c647f69b602 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:42.990294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:43.002692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:43.023667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:43.024228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:43.024515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:43.027214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:43.028435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:43.033607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:43.075945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 240/476] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:43.103769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:43.114697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 237/477] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:42 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:43.242810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2fc3a-a2de-4b22-b324-0b70d9be9553 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:43.287793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:43.288482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:43.288770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:43.289044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:43.289818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:43.291304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:43.329446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.sql.core [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 10:27:43.329446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 10:27:43.329446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [parameters: {'id': '6d89e1aff5474186a7e05bb9f44bd5dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 10:27:43.329446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:27:43.350208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:27:43.351184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application yield resource Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise self.value Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application session.commit() Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 10:27:43.352257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.flush() Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application compat.raise_( Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 10:27:43.353145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:27:43.354912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:27:43.355938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [parameters: {'id': '6d89e1aff5474186a7e05bb9f44bd5dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:43.357319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 10:27:43.359116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:43.360717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 241/478] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 21 10:27:43 2022] POST /identity/v3/roles => generated 144 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:43.360843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:43.496610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce4251-2772-43cf-a446-81cb20828032 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:43.553497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:43.554345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:43.554661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:43.554943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:43.572461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6194e11b-3701-44e8-aada-6c2da9ad02a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:43.581854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:43.583697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 10:27:43.584977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:43.589173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 238/479] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:27:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:43.594764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:43.656109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:43.782827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba9f-8fae-4abb-8157-33b581376b28 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:43.834802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:43.835482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:43.835763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:43.836036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:43.836966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:43.837269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:43.838810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1ad2-1658-4913-8479-616f5e1547df None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:43.854496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 242/480] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:44.161353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ca92-7601-47ec-8c84-eb301acc696d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:44.161871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ca92-7601-47ec-8c84-eb301acc696d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:44.161977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ca92-7601-47ec-8c84-eb301acc696d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:44.163644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 239/481] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:44.171722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:44.172357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:44.172357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:44.197281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:44.198872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:44.320042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 243/482] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:44.345317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651ca92-7601-47ec-8c84-eb301acc696d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:44.386493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:44.489248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651ca92-7601-47ec-8c84-eb301acc696d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:44.543448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:44.544713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:44.545131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:44.545534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:44.571670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:44.586126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:44.587143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:44.588075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:44.592424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 240/483] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:44.600230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:44.639817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:44.768809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e13848-9a45-428d-afa8-1e0ae39930f4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:44.840496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:44.841616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:44.842098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:44.842658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:44.843712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:44.844225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:44.845816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:44.859535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 244/484] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:44.865299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:44.999083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:45.009052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b68fd-08f9-410b-b449-5ccddcbadb61 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:45.048884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:45.049556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:45.049842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:45.050122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:45.063708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:45.065175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:45.093090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:45.094022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:45.094944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:45.094944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:45.094944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:45.094944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:45.095750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 241/485] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:45.103017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:45.376725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:45.434025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa142-1163-4077-bdc3-e8957f20eae2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:45.529889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:45.531118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:45.531609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:45.532081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:45.533238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:45.533952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:45.535892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078887e7-0a52-4c75-a056-19441d8513bd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:45.562203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 245/486] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:45.567545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:45.726709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f43a-83d5-417d-a06d-dbb38034aff4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:45.756706 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f735-4048-433e-b4f7-244f3f10d925 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:45.757021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f735-4048-433e-b4f7-244f3f10d925 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:45.757253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f735-4048-433e-b4f7-244f3f10d925 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:45.758967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 246/487] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:45.765936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:45.766217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:45.766490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:45.787852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:45.788701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:45.789608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:45.789967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:45.799927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:45.801381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:45.803562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:45.804764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:45.821152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:45.822121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:45.823889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 242/488] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:27:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:45.829489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:45.832282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:45.967502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 247/489] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:27:45.992446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0a1-2735-416e-9b50-6ac0bad07a75 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.007720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.055363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.056039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.056318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.056590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.058162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:46.058866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:46.064870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:46.131163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.143707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 243/490] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:27:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:46.151069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.203604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166720-6cae-4855-8116-43a3046e35e9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.267077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.282952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.283773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.284051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.284323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.298754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a7db8485-e14d-4827-ace7-6486da218695 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:46.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:46.307685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:46.308623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 248/491] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:46.314585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b99f-ca42-4fd7-b561-118034055a9a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.314585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b99f-ca42-4fd7-b561-118034055a9a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.314585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b99f-ca42-4fd7-b561-118034055a9a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.316768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 249/492] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:46.317582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4b99f-ca42-4fd7-b561-118034055a9a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.326897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999a964-6b2a-4cfc-add6-dbe4ce1858a6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.364044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.364729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.365009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.365284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.460779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:46.461825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4b99f-ca42-4fd7-b561-118034055a9a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.462345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806c650-ad63-4633-b58f-1bd81823ffee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:46.495421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.504055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.504741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.505021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.505299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.506062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:46.506374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:46.507817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79311421-90a6-4f50-a9f5-6bc458322209 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:46.521633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 250/493] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:46.521633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.521633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.521633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.534755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.542029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:46.543298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:46.592494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 244/494] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:46.594673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189795-b850-4615-98fc-e73a2d5e7faa None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.594979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189795-b850-4615-98fc-e73a2d5e7faa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.595229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189795-b850-4615-98fc-e73a2d5e7faa None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.598593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 245/495] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:46.598593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7189795-b850-4615-98fc-e73a2d5e7faa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.647108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 251/496] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:46.648072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.734913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7189795-b850-4615-98fc-e73a2d5e7faa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.770448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9473d-edad-4363-b0a6-c93da0b7fa8f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.774134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.774731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.775041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.775323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.787456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:46.789357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:46.797891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-98ef9498-b17c-48d1-a19d-f4843e645298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:46.809835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:46.810790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:46.811640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:46.811640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:46.811640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:46.811640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:46.811858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.811858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.811858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.811858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.815879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 246/497] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:46.817144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.817442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.817719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:46.824882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:46.826378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:46.835233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:46.836474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:46.845200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:46.845752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:46.846786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:46.847649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:46.847649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:46.847649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:46.847649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:46.847649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 252/498] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:46.847915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.910661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 247/499] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:46.911514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:46.959977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2f175-3540-4fc1-b97e-f3ec48c4b9b7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:46.997620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:46.998413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:46.998720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:46.998987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.009216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:47.014316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27891fa-36ea-4d10-9b27-f6c4ebca7326 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a77c88d7-522f-4916-b431-68477af42168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:47.017834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.019117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:47.019924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 253/500] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.020856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab101-1ed4-4997-9a36-13735912bda7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.021038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab101-1ed4-4997-9a36-13735912bda7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.021311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab101-1ed4-4997-9a36-13735912bda7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.022877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 254/501] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:47.023533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab101-1ed4-4997-9a36-13735912bda7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.052200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.052770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.053046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.053324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.054077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:47.054378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.055793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.061123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.068253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 248/502] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:47.069364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.072225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.139999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab101-1ed4-4997-9a36-13735912bda7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.177628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.178293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.178683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.178945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.179672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:47.179977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.181350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.183470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa093-e894-440e-a570-fc9c45708cec None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.199659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 255/503] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:47.199659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.217743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.218567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.218850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.219123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.230206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-168859d9-7edd-44a2-91c4-08bf45e55dba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:47.238034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.239128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:47.239957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 249/504] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.240608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.240872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.241097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.258952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:47.260868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:47.302647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.309663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00a14-6c8e-456e-bfc3-f03e97778f2f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.320234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.330428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 250/505] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:47.331300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.346500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.347194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.347505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.347741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.348519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:47.348827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.350205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.360901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 256/506] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:47.361750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.447077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c668d7-7c63-4935-8712-c869cda76928 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.469496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6393-6fb3-47c8-b7d2-d07c16a7adbc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.484401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.485204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.485490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.485768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.493155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.494730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.505406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.506209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.506494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.506793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:47.509970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.516560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 251/507] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.517716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.518484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.518484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:47.531680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.532921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 257/508] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.534264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.553299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.573734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.625817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914785a3-9025-4dc6-a7cd-28b4abde20cb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.645603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f970e-6713-4fec-b020-2f7b5c64f03c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.667000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.668138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.668614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.669112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.670261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:47.670826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.673366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.684251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.685047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.685194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 252/509] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:47.685320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.685586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.685956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.699033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.700688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:47.720435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.721428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.722267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:47.722267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:47.722267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:47.722267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:47.722267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 258/510] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.722528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.796397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.813187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa098e51-005f-4ee7-8ade-c32089813a66 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.831178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:47.831511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d810421-ffe6-4b2d-9768-3d8d9ba6b16b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:47.852326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.852868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.853188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.853464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.864888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.866406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.867844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:47.868522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:47.869002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:47.869141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:47.870673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:47.870991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:47.872717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:47.886261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:47.887363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:47.888195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:47.888195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:47.888195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:47.888195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:47.888195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 253/511] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:47.888463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:47.910496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 259/512] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:47.911420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.006847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643ebb9-fecc-4b52-938f-cdac991320cd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.019222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dffcc-1a3d-477f-bece-2bb977bca075 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.043498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.044158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.046318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.046318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.046318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:48.046318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.047230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.049385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.058178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.059207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.059631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.059631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.068155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 254/513] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:48.068937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.076826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.078312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:48.098456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:48.099392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:48.100255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:48.100255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:48.100255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:48.100255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:48.100463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.100534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 260/514] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:48.101046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.182818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ec76-8c0f-4822-8d0d-459a49f6801b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.207870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ab1a-51dd-41ef-ac6b-56521a9de91e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.219392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.220056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.220331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.220601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.221328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:48.221624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.223046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.235800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 255/515] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:48.236653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.245372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.246090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.246374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.246680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.247807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:48.248114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.249501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.261857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 261/516] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:48.262732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.303332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.334010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.348781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab8b6-b32f-47ce-81d2-c2775af161f7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.370090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52d81-be28-48f2-874d-256756771ed9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.384331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.384975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.385316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.385818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.406906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.407567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.407850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.408132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.408962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:48.409274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.410940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.416376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 262/517] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:48.417806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.472735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.474263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.582049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec49095-016f-42c7-a45d-a3b277e5b927 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.584082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.607765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.628542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 256/518] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:48.629588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.631669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.632384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.632690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.632928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.633734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:48.634298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.636464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.649172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 263/519] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:48.650017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.770538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1296-126c-464a-91e0-f19aa4a9fa80 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.770781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04455f3-abe7-468a-938b-836152ef4bbf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:48.814475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.815878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.816365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.816843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.819359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:48.820205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:48.820492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:48.820764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:48.822305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.824300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.830639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:48.833331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:48.854503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:48.855682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:48.856552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 264/520] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:48.857116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.886432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.918586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 257/521] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:27:48 2022] POST /identity/v3/users => generated 247 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:27:48.918586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:48.984078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:48.991710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd96d-8f99-4661-ba55-769570872555 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.047802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.051219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.051793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.053302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.067701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.068068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3624a-1fac-430d-94a4-928bda553042 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.070320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:49.093750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:49.094717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:49.095569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:49.095569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:49.095569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:49.095569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:49.097688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 265/522] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:49.102856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.111241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.111976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.112262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.119991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.130397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.131905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:49.150725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:49.152121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:27:49.153068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:49.153566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 258/523] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:49.155096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.217071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.254979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5baf2a-c68c-4e7a-a120-8926b1e63234 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.295143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b3586-8782-4909-b4b3-bae925b305df None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.297506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.309633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.310667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.310953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.311273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.312806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:49.313118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.314779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.336318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.337657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.338335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.338982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.340160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:49.340615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.342478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 266/524] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:49.343302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.343434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.363922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 259/525] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:49.365412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.500312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6aa5-2d4a-4a71-8517-a8849eabdd71 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.510613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298875-2dac-406d-a10f-a52bd2ba0950 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.543684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.549203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.549987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.550344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.550714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.551673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:49.551983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.553507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.557026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.558340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.558621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.559179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.579384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 267/526] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:49.579384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.601879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.649888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.652408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.693063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc11a-4989-4a26-9906-7b7825043461 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.733333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.734222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.734507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.734801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.744837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.747637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:49.767489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:49.768394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:49.769239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:49.770160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 268/527] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:49.771642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.785923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 260/528] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:49.787010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:49.834191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.858131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:49.892821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146c7b8-a854-424f-be9c-37dc1c38064d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.904195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f61e-a60c-46a9-8170-83c7318c2f49 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:49.930514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.931192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.931470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.931740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.944103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:49.945152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:49.945581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:49.946034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:49.947046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:49.948172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:49.950460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:49.964799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 261/529] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:49.970785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.011479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.012751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.067109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.083638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a0be-4e88-47b9-9b7f-4ac7319e58c0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.108768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.123598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.124704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.125138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.125564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.130573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 269/530] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:50.130573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.142631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.144367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:50.164941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:50.165975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:50.167493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:50.167493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:50.167493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:50.167493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:50.169099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 262/531] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:50.175276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.249124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61beeb3-da69-4e07-9049-8567a80953bf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.286623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.287454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.287744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.288089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.288985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=1ec420ff94214b3ab6a041395c0a491c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:50.289483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.291022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.291906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0921b79-0c97-461d-899b-075ae69c5e76 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.306581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 270/532] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/role_assignments?user.id=1ec420ff94214b3ab6a041395c0a491c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:50.306581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.336958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.376074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.377137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.377600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.378044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.402741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.404445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.419806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099c71-fe6d-40df-96cd-f6b92893c421 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.457083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.457750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.458028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.458298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.459071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:50.459374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.460738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:50.474598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:50.476510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:50.478411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:50.478411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:50.478411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:50.478411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:50.479562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 271/533] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:50.481025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.482428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 263/534] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:50.488701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.550574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.590947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.661528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98931ff-e977-4863-ad7d-ccd7975a5c25 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.688680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ac74e-4511-4bad-a48f-c53b567a2802 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.730211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.731098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.731386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.731659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.745994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.747866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.751810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.752488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.752765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.753036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.753778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:50.754067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.755532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164dc5-ab15-476b-ad2c-128ccd77c2cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:50.776157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 264/535] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:50.776157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fedd-861c-43a5-b82c-dd08961a995b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.776157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fedd-861c-43a5-b82c-dd08961a995b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.776157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fedd-861c-43a5-b82c-dd08961a995b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.776157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 265/536] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-38463553-abda-4f61-a74f-2dcb0eda2df7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:50.777268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:50.778103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:50.779004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 272/537] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:50.779471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9fedd-861c-43a5-b82c-dd08961a995b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.788599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.789244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.789728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.827437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:50.830262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:50.909581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:50.928558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9fedd-861c-43a5-b82c-dd08961a995b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:50.976935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:50.977777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:50.978125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:50.978488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:50.986707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 273/538] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:50.986707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:50.993223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:50.995076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:51.022674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:51.023575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:51.024412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:51.031985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 266/539] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:51.031985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.098798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.153544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35014f9-d7d0-49f5-9832-06b8979e8ce4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.176915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e258415-5688-4730-aeeb-9e733f268afb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.205467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.206161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.206443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.206767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.207627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:51.208493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:51.210854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.222081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.223322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.223915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.224397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.242655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 274/540] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:51.242655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.242884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:51.254203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:51.255477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:51.256794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:51.261424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 267/541] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:51.262841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.276233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.383506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775fcb8-b88c-414a-bd7d-1a08e80831db None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.393657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76380b3-a036-40e1-b941-52602c3bd85e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.409544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.420413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.421107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.421360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.428111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.428111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:51.428111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:51.428111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05564270-e5b5-40bc-b816-519e21133941 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.434959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.446579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.446579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.446579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.447138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 275/542] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:51.447318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=ebd9b2da29ee4612afb617183145553b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:51.447585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90851-021c-4b32-a736-80b163f197a7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.448238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:51.448390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90851-021c-4b32-a736-80b163f197a7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.448390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90851-021c-4b32-a736-80b163f197a7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.448655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fb37-3db1-47c0-bf36-84d2031db389 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.448819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 276/543] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:51.448819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90851-021c-4b32-a736-80b163f197a7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.458614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 268/544] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/role_assignments?user.id=ebd9b2da29ee4612afb617183145553b&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:51.464599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.464980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.465303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.490609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:51.492332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:51.580965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.591289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 269/545] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:51.592120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.629635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90851-021c-4b32-a736-80b163f197a7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.679041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.680100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.680465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.680809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.681631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:51.682063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:51.684091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.698954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 277/546] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:51.700110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.767892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b94c-fd0e-4ce6-8740-d63182d91eca None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.785705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.828387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.829352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.829930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.830417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.831500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=dbd2ab88b240473aa0067856ec2254f8) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:51.832037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:51.833873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:51.841288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 270/547] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/role_assignments?user.id=dbd2ab88b240473aa0067856ec2254f8&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:51.842632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.855937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f143e-9c7b-47b2-a6d6-5c817eb75507 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:51.923435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:51.928182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:51.929215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:51.929424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:51.929770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:51.953547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:51.969760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:51.971100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:51.973082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 278/548] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:51.978159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:51.983232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5aec-a8fa-44c3-b7b9-67779d53e1df None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.026503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.027923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.028406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.028864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.045841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.047790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:52.067852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:52.068815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:52.069674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:52.069674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:52.069674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:52.069674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:52.072210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 271/549] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:52.078475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.207739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43717614-dac2-4d16-a7ed-0530788460ea None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.219025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:52.248872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.250082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.250880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.251351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.252302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:52.252904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.254662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.265539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 279/550] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:52.274223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.413648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e392-46dc-43ce-8dae-946057a17a5f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.422034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d436-0bd6-44d2-9b28-89cb3710abf6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.470710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:52.521463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.522185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.522468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.522788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.523698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:52.524077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.525486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.536500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.538142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.539033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.539745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.540670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 272/551] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:52.548805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.561013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.563636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:52.586178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:52.587269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:52.588228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:52.588228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:52.588228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:52.588228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:52.590943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 280/552] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:52.598887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.645771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:52.733259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431363b-9072-4be2-9df0-35b46d7f1b82 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.803534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce5e96-e8be-4760-ad18-4f53a72448ba None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:52.844835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.846170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.846676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.847168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.850914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:52.852038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:52.852511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:52.852980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:52.854448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:52.855030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.857625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.866570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:52.866570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:52.908197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 281/553] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:52.914374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:52.921737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:52.922854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:52.923777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:52.924310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 273/554] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:52.935113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:52.998374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.112201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d796-293a-44d1-983a-6def7b964ca7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.130170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.167888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07564e-958d-4333-b05c-86311a51be5b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.169753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.171055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.171555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.172032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.173210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:53.173742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.175446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32914d-9ce6-41f8-8102-4edf99976781 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:53.207416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 274/555] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:53.209267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037189a-fd16-41b9-b6e5-b62e31ae9828 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.209758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037189a-fd16-41b9-b6e5-b62e31ae9828 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.210260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037189a-fd16-41b9-b6e5-b62e31ae9828 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.212003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 275/556] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:53.215343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037189a-fd16-41b9-b6e5-b62e31ae9828 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.243029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.243958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.244246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.245324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.254280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.255715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-09912ec1-5c7c-4a7e-8fbe-3babb4ab38fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:53.273038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:53.275079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 282/557] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:27:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.278479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.280993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.307516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:53.309782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:53.348227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.357004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037189a-fd16-41b9-b6e5-b62e31ae9828 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.397222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.399013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.399544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.400333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.405274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 283/558] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:53.406725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.420897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.423283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:53.447735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:53.449207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 276/559] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:53.451537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.503604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.542886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9baca7a-e1d1-46ef-8260-3f62c0eab11c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.592823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.593534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.594218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.594504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.595679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:53.595995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.597446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:53.597855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa228-c98e-48ab-8bee-25b2701b0073 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.622626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 284/560] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:53.623817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.648984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.649788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.650070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.650374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.665247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:53.675718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:53.676655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 277/561] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:53.678146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.701362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.779244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c405ca5-63dd-4afe-93b9-47aea2422996 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.825633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:53.835213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.836159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.836604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.840183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.840183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:53.840183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.840183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:53.844721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d941-7e7c-4cf0-8c26-a394c9c9b3c6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.853477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 285/562] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:53.855258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:53.898817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:53.900659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:53.902229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:53.902815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:53.979972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35042-8c51-47e0-a01f-980e8474b954 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:53.997396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:53.999014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.010994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.021028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.022965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.022965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.022965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.023347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:54.023713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.025093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.035849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 286/563] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:54.036609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.105027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.132053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 278/564] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:27:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:27:54.134744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.180084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54cc2-da3e-430e-80e1-c9030d447983 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.233165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.233975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.234261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.234525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.247293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.248781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8666aa5d-7be2-454a-bf41-1bbcb470b438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:54.264316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:54.265288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:54.266163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 287/565] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:54.267409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518cf6-9127-486b-86e2-62cf0bcb724c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.267409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518cf6-9127-486b-86e2-62cf0bcb724c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.267409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518cf6-9127-486b-86e2-62cf0bcb724c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.268812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 288/566] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:54.270696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518cf6-9127-486b-86e2-62cf0bcb724c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.282532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab567-ce21-48af-afdb-ba0d336c3c62 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.293541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.324885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.325737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.326040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.326319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.340177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.341666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:54.371521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:54.372733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:54.373614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:54.373614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:54.373614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:54.373614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:54.374353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 279/567] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:54.375480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.402012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518cf6-9127-486b-86e2-62cf0bcb724c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.414493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.445197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.446459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.446952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.447489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.465325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.466837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d825b65f-4348-4a29-861d-3d8b408c211f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:54.486840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:54.488816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:54.490495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:54.490495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:54.490495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:54.490495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:54.492775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 289/568] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:54.494124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.495096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.495569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.514424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:54.515651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:54.520557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974585a-9a59-4608-97fb-c14276fb8f74 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.561295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.562455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.562765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.563041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.563886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:54.564194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.565990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.582472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 280/569] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:54.583559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.593749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 290/570] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:54.594638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.607331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.696440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4abb4-a64a-40af-b928-cb44f436df41 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.701637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.716890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679c53-dcb2-4746-ac8c-f05e706a2128 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.739098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.739765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.740048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.740342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.741054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:54.741353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.742772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae37f48-f0dc-42b0-8ca9-7b62b52787d7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.756847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 281/571] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:54.757991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723da584-9546-4ae4-9f98-35ed2d2bc470 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.758244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723da584-9546-4ae4-9f98-35ed2d2bc470 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.758516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723da584-9546-4ae4-9f98-35ed2d2bc470 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.760084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 282/572] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:54.760736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723da584-9546-4ae4-9f98-35ed2d2bc470 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.766948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.767627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.767908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.768182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.768919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:54.769227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.770665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.784038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 291/573] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:54.784901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.851597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:54.885000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723da584-9546-4ae4-9f98-35ed2d2bc470 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.896946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f216-1037-458e-ac20-76b47d2f0026 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:54.919121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.920009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.920289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.920562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.930917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:54.931704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:54.932710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:54.933165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:54.935992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=23fabbe0e78d45f890b21fcf4b348721, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:54.937437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1d4ab-1d1d-4483-a849-d0f3cb69eae3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:54.943548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:54.970795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:54.971972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 292/574] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:54.973014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:54.973921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.014045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 283/575] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:54 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:55.015364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.015653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.015932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.035607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:55.036999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:55.091838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443082e8-b136-4f88-9d07-91be84a09968 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.139621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.140724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.140998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.141258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.150307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.151610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 284/576] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:55.152579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.152808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:55.180877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:55.182851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:27:55.184233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:55.184767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 293/577] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:55.186317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.254800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.288546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.301972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4b8e-c77d-43d5-a61b-e1405e1b54ab None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.330814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b14de-0d5b-4432-a501-91794dadba8b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.340197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.341024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.341316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.341594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.349361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.351022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:55.367093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:55.375145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 285/578] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:55.376043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.386978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.388086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.388646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.388932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.389909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:55.390224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.391964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.401345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 294/579] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:55.402523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.495925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ddc7c-e003-483c-a7b4-e6565190d90b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.528745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971e4aa-dd00-4fed-a6a2-0863bb369d8d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.535383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.561395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.562206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.562496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.562810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.574668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.574966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:55.575857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.576358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.576852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.577898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:55.578429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.580128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.582397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:55.582672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:55.583570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:55.584482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 286/580] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:55.585169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.605202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 295/581] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:55.610727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.703963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7632c-0506-4f6a-92b5-8ea2f2a1a1e7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.729853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8ab5b-227d-4e29-ad30-ed94d6c159dd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.742398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.743174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.743519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.743805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.745327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:55.745680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.747472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.762323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 287/582] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:55.767927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.770182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.771026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.771312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.771591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.783325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.784705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:55.803811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:55.805013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 296/583] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:55.805971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.820871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.865211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:55.899258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efdb8e-0cd8-4867-9691-4333b2e7c2d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.930127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fec9fa-b1df-4631-9417-0af159838345 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:55.944678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.945575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.945716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.946015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.946749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:55.947049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.949039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:55.962913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 288/584] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:55.962913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:55.975282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:55.976189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:55.976473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:55.976750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:55.991116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:55.992712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:56.019446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:56.020407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:56.021292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:56.021292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:56.021292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:56.021292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:56.029542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 297/585] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:56.031055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.075929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.079645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fe597-1a0b-4c7b-8e05-1e8cce35237d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.116224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.117032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.117358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.117823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.118978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=7b4c6a5adbda4403883661e072aba906) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:56.120765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.122772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.132528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 289/586] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:27:55 2022] GET /identity/v3/role_assignments?user.id=7b4c6a5adbda4403883661e072aba906&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:56.133379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.206074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.217764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba4cd-2f18-4f90-8967-cc89fff931ef None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.272265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.272975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.273264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.273835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.274618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:56.274934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.277300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.299313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 298/587] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:56.303589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.311544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a29-65dc-4e11-b871-50f1e66832da None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.354534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.355454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.355761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.356041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.367038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.368471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:56.392194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:56.393187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:56.394240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:56.394240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:56.394240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:56.394240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:56.400165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 290/588] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:56.401986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.430010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.462730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e7e36-268a-421c-be48-1a4639c4fefb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.517792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.518468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.518774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.519053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.519780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:56.520084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.521625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.539906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 299/589] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:56.539906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.565018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.586252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555685-6230-480a-868b-f263ca4e7767 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.639379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.641738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.642759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.643259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.659484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.661004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.678183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d4aa-38f1-407a-ac71-233fa855f0e0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:56.683084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:56.684154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:27:56.685089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:56.689536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 291/590] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:56.690568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.720857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.721925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.722274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.722668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.723490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:56.723872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.725466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.746193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 300/591] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:56.746193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.798999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.834425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df74130-801e-49f9-8043-90accfcafc7a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.851520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:56.879459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b9972-f7dd-4f57-9a09-699203a3ed30 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:56.882366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.884523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.885562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.886213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.896116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.898130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:56.920262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:56.921389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:56.922486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:56.923977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 292/592] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:56.924805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:56.930112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:56.931092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:56.932219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:56.932991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:56.934111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:56.934664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:56.936415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:56.965185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 301/593] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:56.966509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.062925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd0ea-d33e-481f-8778-421c44a1ae37 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.091495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cb8a2-7eb7-4397-89e3-cdf89d7c9a6e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.108851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.111211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.111959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.112483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.118464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.125139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.127184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.138790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.139642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.140071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.144018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.144018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:57.144018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.144018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.145156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:57.148025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:57.148947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:27:57.150035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:57.157931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 293/594] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:57.159278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.161615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 302/595] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:57.162533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.297856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9c883-8352-4e94-b556-dc35d90e24ee None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.305192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f2a39-28e6-4405-b86f-19211e3ea7e2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.350383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.350899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.351892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.352024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.352317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.352728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.353141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.353693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.356065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:57.356603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.359240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.367962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.371953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.384334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 303/596] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:57.386411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a029bcc8-ebef-4563-a2be-73e6d4bdb9d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:57.401315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:57.402533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:57.403466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:57.403466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:57.403466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:57.403466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:57.407726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 294/597] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:57.408379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67747df-2796-4aa3-9882-72d13cf8db62 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.408666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67747df-2796-4aa3-9882-72d13cf8db62 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.408943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67747df-2796-4aa3-9882-72d13cf8db62 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.410350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 295/598] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:57.413790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67747df-2796-4aa3-9882-72d13cf8db62 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.467834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.488872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.519260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84b58c-7514-40d9-a86b-438a409a9712 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.559151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.560060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.560337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.560612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/1ec420ff94214b3ab6a041395c0a491c/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.569268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67747df-2796-4aa3-9882-72d13cf8db62 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.579917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=1ec420ff94214b3ab6a041395c0a491c, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.581651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b124d-389c-4908-a3a8-7cc620dbcef9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.611605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.612385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.612798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.613090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.625380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.639875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:57.645616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:57.646746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 296/599] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:57.647802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.677810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 304/600] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:57 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/1ec420ff94214b3ab6a041395c0a491c/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:57.679772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.680344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.680932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.707520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:57.709669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:57.789719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.793350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f1d7-a064-4c53-9ed7-af8c42cd123f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:57.824127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 305/601] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:57.825112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.857795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:57.859233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:57.859734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:57.860220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:57.862044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:57.862639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:57.864914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:57.869444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:57.883160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 297/602] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:57.884516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:57.980905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7568311-18cd-4fa7-af36-c0248aace0a4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.030167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.031249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.031700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.032153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.035419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:58.036267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.037734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.054253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 306/603] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:58.055171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.084852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60315586-8dd8-40cb-a041-04f0e0bf82d1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.125035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.126431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.126923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.127351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.146385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:58.169267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:58.170937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:58.171786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:58.175764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 298/604] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:58.176719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.185350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.195558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ea31a-bbfd-4295-a29d-f19e7bbaf786 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.222146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.238416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.239654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.240143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.240628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.249816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.254770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:58.283019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:58.283955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:58.284906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:58.287594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 307/605] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:58.289015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.355537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f98ca4-f817-4714-82d9-1c013bd73fa6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.405073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.406710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.407216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.407709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.421790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.424182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.437435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01647f-1e6a-43a4-84c7-ae67bae40e6c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:27:58.449458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:58.450690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:27:58.451733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:58.452224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 299/606] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:58.452948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.482823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.483786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.484142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.484398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.485103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:58.485412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.486853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419e4b3-b31c-43b8-99bf-8aba18a1b73b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.498514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 308/607] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:58.499824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af3a5-fff2-4667-8ee5-3b7a84d95607 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.500118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af3a5-fff2-4667-8ee5-3b7a84d95607 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.500398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af3a5-fff2-4667-8ee5-3b7a84d95607 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.501792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 309/608] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:27:58.503331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af3a5-fff2-4667-8ee5-3b7a84d95607 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.532292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.571058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.594525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ab-b94a-46d1-b71a-3dacc7186076 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.647061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.648347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.648834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.649309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.650759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:58.651498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.654073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce2065-328b-48a9-82ad-ab38b0f6ee9f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.658499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af3a5-fff2-4667-8ee5-3b7a84d95607 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.669158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 300/609] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:58.670614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.670909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.671188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.692160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:27:58.692160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:27:58.714313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.715174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.715452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.715725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.716542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:58.716830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.718307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.731401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 310/610] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:58.734761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.778670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 301/611] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:27:58.778670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.847276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.881409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fec8-98eb-42f4-bf3a-25ff9f8827a7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.885580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:58.923635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b0c52-9d4b-4825-af5d-a4efaf891b16 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:58.928357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.929284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.929562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.929832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.941794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.943321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:58.964530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:58.965593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 311/612] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:58.966603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:58.970953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:58.971856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:58.972261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:58.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/ebd9b2da29ee4612afb617183145553b/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:58.988577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=ebd9b2da29ee4612afb617183145553b, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:58.990029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.083347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 302/613] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:58 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/ebd9b2da29ee4612afb617183145553b/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:59.084422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.112081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c1b-0646-4d70-9902-567fff2ac9c0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.151752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:59.169292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.170702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.171171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.171612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/dbd2ab88b240473aa0067856ec2254f8/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.209530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=dbd2ab88b240473aa0067856ec2254f8, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:59.211497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.254779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bac08b-6173-436b-b859-e05d418d6869 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.283393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:59.291910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 312/614] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:27:58 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/dbd2ab88b240473aa0067856ec2254f8/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:27:59.293101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.300219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.301357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.301706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.301987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.317066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:27:59.325790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:59.326859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:59.327705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 303/615] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:59.339375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.486064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f15f-ec4f-4de9-ab5b-54c1fe8b4234 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.516076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bce9a-514b-43f2-8ede-6f178d956983 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.537735 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.539164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.539599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.540025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.540909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:59.542145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:59.544153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.557264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.557950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.558236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.558646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.559471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:27:59.559631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 313/616] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:27:59.559736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:59.561078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.567893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.575836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 304/617] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:27:59.581333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.611118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:59.629727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:27:59.708646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687955f-c93d-4345-bae8-35b03d6298a2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.739055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a8c8d-9c1a-4e0d-b015-47b9e07ac14d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:27:59.780620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.781512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.781797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.782074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.809301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:27:59.810132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:27:59.810474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:27:59.810779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:27:59.812181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:59.813647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.833129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:27:59.834657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:27:59.847358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:59.849371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:59.851190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:27:59.851190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:27:59.851190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:27:59.851190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:27:59.852577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 305/618] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:27:59.859237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:27:59.870745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:27:59.874344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:27:59.876712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 314/619] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:27:59.883359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.048342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.057460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbaeec-d03f-4a73-a665-e9f3a05f265b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.089266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.095629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe59e6-8d81-4b53-82b6-aba221431b10 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.099368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.105794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e5d9a-e102-4e40-ad79-ae8a3d1f5b3e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.120250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 306/620] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:00.128237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754bf7-0186-465a-905f-5b757fd4a875 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.129000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754bf7-0186-465a-905f-5b757fd4a875 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.129638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754bf7-0186-465a-905f-5b757fd4a875 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.131936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 307/621] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:00.133000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754bf7-0186-465a-905f-5b757fd4a875 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.146172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.146921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.147684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.147967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.148821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:00.149128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.151719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32289a-6e19-4710-ad97-182a6e59d88c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.178340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 315/622] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:27:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:00.178600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.178820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.179024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.205647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:00.207192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:00.284006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754bf7-0186-465a-905f-5b757fd4a875 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.302427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 316/623] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:00.303645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.340157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.341306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.341791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.342250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.350428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.352716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:00.369831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:00.370940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:00.372173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:00.374492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 308/624] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:00.380095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.397728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.421897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.443914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99e328-7b44-4b32-b882-728788240da7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.499953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.501532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.502001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.502467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.531181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abf955-a594-47a0-a0c4-d523c53dd4b3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.571240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.572364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.573930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.574466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.586767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:00.590467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.592064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:00.596497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:00.597613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:00.598580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:00.600862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 309/625] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:00.601817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.691285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.710090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.720307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77df3c-5083-45f4-966f-8d4c4e6bbeac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.730971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 317/626] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:00.732169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.760851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.761666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.762065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.762452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.763592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:00.763956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.765676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.786805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 310/627] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:00.787882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.854875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f55a-0443-43dd-b0e6-e46ede38c550 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.897944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.899323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.900031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.900615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.902211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:00.902705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.904518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.905093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414f8d-833a-4dbc-b27f-ec8d6fb880bc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:00.916799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 318/628] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:00.918369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:00.943396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.949533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:00.950608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:00.951006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:00.951373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:00.964603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:00.966623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:00.982798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:00.990485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:00.998996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 311/629] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:00.999913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.077462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c43cf1-8cfa-4fde-8fcc-c6d83fd84985 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.130478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958de1-1028-4dda-97b2-2f8c1bebd848 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.131329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.132283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.132602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.132895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.148013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.149594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:01.177335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:01.178292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:01.179336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:01.179336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:01.179336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:01.179336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:01.179590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.179590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.179590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.179856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.182169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 319/630] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:01.183920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.195959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.197453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:01.224745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:01.225703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 312/631] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:01.227235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.279203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.302694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.307093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a9e0d-2344-41ba-9cd6-480124cbb3a3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.343743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.344875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.345374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.345964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.346916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:01.347448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.349010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.362730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 320/632] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:01.363945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.375965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c643cda-fe58-4882-a4ed-a9a23d9f0139 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.426858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.427585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.428021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.428435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.429232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:01.429542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.431857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.450063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 313/633] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:01.450915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.490327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb99d-24bc-4fc4-a977-b78aa8028f95 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.547319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.548538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.548835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.549114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.549881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:01.550194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.552181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.569357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 321/634] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:01.570350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.576284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.584281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d288-4226-49e6-b582-23e465ed9c70 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.593637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.620472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.621266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.621805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.622087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.629098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.630942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d54e8ad1-fb74-4fde-9d24-c65f18be2146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:01.647708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:01.648711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:01.649602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 314/635] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:01.651333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d8bbf-f33d-457f-a2f5-9733575fb0c1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.651333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d8bbf-f33d-457f-a2f5-9733575fb0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.651520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d8bbf-f33d-457f-a2f5-9733575fb0c1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.657005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 315/636] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:01.657005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d8bbf-f33d-457f-a2f5-9733575fb0c1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.703983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731210b-3e72-4923-9512-7414e9d12859 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.756823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.758044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.758450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.758935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.770571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.773054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.787371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d8bbf-f33d-457f-a2f5-9733575fb0c1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:01.795748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:01.797870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 322/637] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:01.799958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.834476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.835310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.835594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.835875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.843878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.845394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-21a05618-663a-4be5-b6a7-e84f2da0f9ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:01.860638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:01.861729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:01.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 316/638] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:01.863399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.863399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.868748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.881418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.881613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:01.882658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:01.905228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:01.905479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24181-ba2f-4c8c-8637-76e67c131183 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:01.943454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:01.944227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.944511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.944787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.945579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:01.945896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:01.947956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71e5c5-cf3c-4d77-b36c-6eb761b8e92e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:01.960547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 323/639] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:01.962135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18a7ca-ff10-4460-a16d-b72d09dd6c52 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:01.962420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18a7ca-ff10-4460-a16d-b72d09dd6c52 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:01.962733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18a7ca-ff10-4460-a16d-b72d09dd6c52 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:01.968593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 324/640] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:01.968952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 317/641] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:01.969389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18a7ca-ff10-4460-a16d-b72d09dd6c52 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:01.969793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.083311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294297f0-2cbd-4eec-85dd-b7e562839f08 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.091138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18a7ca-ff10-4460-a16d-b72d09dd6c52 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.122149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.123112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.123534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.123936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.124931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:02.125328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.127093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b428c9f-d0fd-4697-9362-b2e8bb9e37b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:02.127463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.128180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.128518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.128799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.129640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:02.129949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.131486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:02.139849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.145359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 325/642] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:02.146378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.148409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 318/643] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:28:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:02.149962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.150342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.150657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.156605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.168086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:02.169445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:02.240574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 319/644] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:02.242590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.270685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7a763-cd53-41d5-b0e8-b8066c27cacc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.307360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.308014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.308291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.308564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.309422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:02.309733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.311238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:02.317532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 326/645] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:02.318352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.366649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece63e-2a70-4846-b3ab-a9a93f34597f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.409856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.413430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.414244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.414508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.414817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.418005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.429381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:02.438244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:02.439578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 320/646] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:02.446266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.468142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a180e4-81ad-4506-9d0c-26393fda641b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.509633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.511228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.511624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.512013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/7b4c6a5adbda4403883661e072aba906/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.531104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=7b4c6a5adbda4403883661e072aba906, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.532516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:02.575602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242547c-7826-4408-9bcd-77c5532c7bab None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.602992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 327/647] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:02 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/7b4c6a5adbda4403883661e072aba906/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:28:02.604836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.628663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.629654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.630193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.631405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.730994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.737590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad078-a954-4513-8399-d58f7d833cee None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.737800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.739158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:02.762675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:02.789027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.789959 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.790236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.790513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.805467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:02.816029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:02.817041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:02.817883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 328/648] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:02.818655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.881094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 321/649] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:02.883679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:02.952616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1c25-7bb0-4156-ba43-1d2a2a19d5e1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:02.994410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:02.995144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:02.995430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:02.995821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:02.996994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:02.997302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:02.998701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.006499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.009567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 329/650] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:03.011195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.020265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1210-f4b9-472a-8670-c8afc9ae5281 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.049879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.063675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.064826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.065241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.065523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.066467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:03.066796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.068469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.085265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 322/651] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:03.086693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.133431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453efd0e-7fae-401b-96e1-037fab862de3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.176235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.177229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.177691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.178071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.194121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.195784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:03.216516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:03.218264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:03.220275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:03.220275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:03.220275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.220275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:03.220275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 330/652] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:03.220855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.225910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4772c-3925-4262-8a0d-60e47e6acb5a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.271890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.272724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.273010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.273278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.293859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.296044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:03.321207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:03.322219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:03.323178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:03.323178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:03.323178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.323178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:03.324009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 323/653] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:03.325088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.345602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.370468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef7326-a9e1-4690-8017-e70ba73d8444 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.416396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.418461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.419433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.419867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.435732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.438331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.451078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb499-a48e-4221-bf53-de4427fb18a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:03.466930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:03.468116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:03.469109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:03.469109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:03.469109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:03.469109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:03.475807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 331/654] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:03.475807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.501420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c017-0680-43de-b36a-579531baa06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.502881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c017-0680-43de-b36a-579531baa06f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.503444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c017-0680-43de-b36a-579531baa06f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.503902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c017-0680-43de-b36a-579531baa06f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.505084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c017-0680-43de-b36a-579531baa06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:03.505554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c017-0680-43de-b36a-579531baa06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.507875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c017-0680-43de-b36a-579531baa06f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.528438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 324/655] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:03.529595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c017-0680-43de-b36a-579531baa06f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.605336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.608032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e065326-1bd3-4d3d-9a16-1b8380b1c7d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.648224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c017-0680-43de-b36a-579531baa06f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.657328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.658451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.658946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.659392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.661170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:03.662130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.665285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.682863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.690027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 332/656] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:03.690907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.691514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.695648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.695648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.695648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.695648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:03.695648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.696686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.710774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 325/657] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:03.712115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.828589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d3ec-c624-4998-b3e6-e67a350e0d52 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.828768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018135b6-a061-4e6d-818a-b54b789f407c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:03.868528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:03.871006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.872410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.872897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.873376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.878048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:03.878875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:03.879154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:03.879421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:03.881995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.889722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.889864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:03.889864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:03.898596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:03.899504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:03.900427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:03.902923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 333/658] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:03.904089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:03.910799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:03.912913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 10:28:03.915621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:03.921201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 326/659] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:03.922341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:03.977033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.022779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7e58e-0204-4073-9c48-9a693707cb80 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.067493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.068688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.068974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.069251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.071598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0faa9-c7fa-42d0-be78-11f32d4378f8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.083265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.085267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:04.110796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:04.112046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 334/660] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:04.113897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.123548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.124462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.124898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.125327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.126930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:04.127402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.129649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.146103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 327/661] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:04.147844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.223942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.258644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac288e7-03c4-4aa1-9f4e-46a19a681980 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.282267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.298679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be103-78aa-4165-8f1a-3ba31925a372 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.302758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.303452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.307211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.307211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.307211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:04.307211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.307717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.333591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 335/662] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:04.334478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.349686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.350778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.351069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.351349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.352503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:04.353907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.355875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.393599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 328/663] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:04.395212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.517442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a7b36-ee64-47ff-8fbb-c75447a8e9df None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.578302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.579444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.579943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.580468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.581991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=1ec420ff94214b3ab6a041395c0a491c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:04.582617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.583797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b902-1c7e-473b-847b-9e3858dc859c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.585260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.596142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 336/664] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/role_assignments?user.id=1ec420ff94214b3ab6a041395c0a491c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:04.597600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.598602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.637457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.638812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.639539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.640108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.691365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.735881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.737551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ba07a-e16c-4bfa-a1f0-3b8a91784175 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.767830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248be06a-8e71-4998-a53f-68c231c0dd82 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:04.812803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:04.813466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.813751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.814030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.855620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 329/665] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:04.856909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d70-a3fb-4f58-8f4e-ec7d5373ade2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:04.857192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d70-a3fb-4f58-8f4e-ec7d5373ade2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:04.857475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d70-a3fb-4f58-8f4e-ec7d5373ade2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:04.863361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 330/666] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:04.868909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632d70-a3fb-4f58-8f4e-ec7d5373ade2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:04.880421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.912964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:04.916647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31549fb6-7408-4301-ae83-00d9b9c2da26 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:04.968365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:04.990747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632d70-a3fb-4f58-8f4e-ec7d5373ade2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.033994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.035085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.035522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.035962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.037463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 337/667] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:05.040808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.041712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.042239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.049707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.050760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:05.062072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:05.063337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1a8f0519-0a8b-4d39-be9a-ec75d0e72858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:05.075057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:05.076022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:05.076917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:05.076917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:05.076917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:05.076917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:05.076917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 331/668] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:05.078739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b060-9d06-4b56-8741-01c4bb1fb013 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.079086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b060-9d06-4b56-8741-01c4bb1fb013 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.079425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b060-9d06-4b56-8741-01c4bb1fb013 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.081701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 332/669] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:05.082902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88b060-9d06-4b56-8741-01c4bb1fb013 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.153696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.154070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 338/670] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:05.155172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.229413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88b060-9d06-4b56-8741-01c4bb1fb013 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.255939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.278318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.279208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.279495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.279769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.291886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.293356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:05.310953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce399878-1be1-4589-bce8-07127d694a73 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3515d0af-cfb7-4848-b03b-ad0cb658d945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:05.312676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:05.313800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:05.314739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:05.314739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:05.314739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:05.314739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:05.314739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 333/671] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:05.315433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.315433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.315709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.332694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:05.333950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:05.360039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.360990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.361448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.361870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.362807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:05.363105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.364493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:05.383494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 339/672] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:05.383494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.421617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 334/673] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:05.421617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.473303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.531983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d503-e910-47a2-b6e5-f8fc99cad665 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.555566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59a084-4f02-4733-b595-1bcdd021a7e9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.569553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.577032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.577905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.578192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.578467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.590004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:05.598063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:05.599240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:05.600352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 340/674] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:05.601159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.601664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.602869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.603427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.604076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.605026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:05.605337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.606787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:05.626993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 335/675] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:05.628184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.736226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ad041-fa2c-4d39-9b41-e2b2eca96de7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.757587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.761998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865163fe-58b0-4bcc-97a0-6f6778530d0a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.781340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.782148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.782433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.782743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.790366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.792138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:05.800444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.801528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.801996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.802453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:05.813192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:05.814135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:05.815105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:05.815555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 341/676] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:05.816331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.818599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:05.827403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:05.828977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 336/677] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:05.830363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:05.887197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:05.951957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3805667-1fc0-4e43-972d-edca8cc4d117 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.969621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3be97-0c5d-4f4f-bacf-3e9a1564085b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:05.988664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:05.989371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:05.989650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:05.989938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:05.997921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:05.999626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:06.014880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:06.016472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 337/678] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:06.017871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.027239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.028231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.028621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.029003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.030084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.030519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.034017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.048575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.054734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 342/679] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.054734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.157709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e783-9ab8-4141-9946-f2208ceeba78 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.178164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceec403-dd70-4b82-966e-fe219945dd10 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.197209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.203062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.203730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.204015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.204447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.205272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.205587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.207044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.219590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.220266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.220547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.225874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 338/680] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.226036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.226118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.226191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.226191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.226191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.235167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 343/681] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.239993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.328035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.349480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a78e-efbc-4dc1-9162-1741ac33f4eb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.362891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096ba5-0df5-41ab-98c0-20e6eb008efd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.389398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.390069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.390327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.390767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.391542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.391845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.393343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.404870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.405694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.410862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.411036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 339/682] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.411232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.411325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.422511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.423986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:06.443192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:06.444242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:06.445242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:06.445242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:06.445242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:06.445242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:06.445242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 344/683] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:06.445556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.479103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.548030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d843827-0eeb-4c47-a1ce-9b9f65ab95f5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.586300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae9655-0813-4eab-8140-47c3c25a53f4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.592337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.593047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.593329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.593569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.594401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=ebd9b2da29ee4612afb617183145553b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.594745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.596189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.614732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 340/684] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/role_assignments?user.id=ebd9b2da29ee4612afb617183145553b&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.614732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.620626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.641305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.642364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.642697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.642978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.668573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.671234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:06.691887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:06.692911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:06.693825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:06.693825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:06.693825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:06.693825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:06.698604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 345/685] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:06.699608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.773942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2976f-632a-4c18-a866-c8c85e87e8ac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.812547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.819038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.819747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.820032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.820329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.821789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=dbd2ab88b240473aa0067856ec2254f8) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.824575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.836398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20c8f9-f312-44e4-bc34-3395fb86c78f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:06.839134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 341/686] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/role_assignments?user.id=dbd2ab88b240473aa0067856ec2254f8&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.839770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.892845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:06.893578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.893945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.894181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.894968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:06.895278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:06.896733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679e097-39fe-4302-bc66-da213166cec5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:06.912363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 346/687] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:06.913857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3f871-d321-4e47-bcde-6f806fa8de8c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:06.914228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3f871-d321-4e47-bcde-6f806fa8de8c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:06.914689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3f871-d321-4e47-bcde-6f806fa8de8c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:06.917032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 347/688] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:06.921967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3f871-d321-4e47-bcde-6f806fa8de8c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:06.946792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:06.995332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6668d-19a8-47c5-a27e-70806be48feb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.039408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.040078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.040363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.040635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.041600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:07.041877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.043277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f33e-76dc-4882-995d-9e139d789e89 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.055636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 342/689] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:07.057431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.057716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.057990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.061307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3f871-d321-4e47-bcde-6f806fa8de8c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.076717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:07.077828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:07.111668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.111668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.111668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.111668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.127962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.129431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.131531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:07.147318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:07.148323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:07.149291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 348/690] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:28:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:07.154588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.176493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 343/691] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:07.177457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.275010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:07.302577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd5da-47a8-418d-aac9-d7ef4d8c2240 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.327405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9743a31-e042-4f4f-a8a3-3a844c9b2045 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.347768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.348833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.349407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.349684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.361529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.363667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.375126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.375969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.376508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.376880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.378059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:07.378510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.380093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:07.386566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:07.387957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 349/692] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:07.389807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.404870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 344/693] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:07.405852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.480749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:07.546172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d14421-9042-4c53-b86c-dbc42142c6c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.597017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.598838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.599277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.599811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.600771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.602685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.612651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d353976-a16c-4623-97a4-72d535a584d2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:07.621178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:07.622898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:07.623745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:07.626935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 350/694] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:07.628003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.644208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:07.669201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.669881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.670164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.670436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.671308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:07.671614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.673234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.695440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 345/695] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:07.696312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.776358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed58fc5-fc44-4055-b930-eaa4c8963302 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.814846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:07.823151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04407-4fe3-43d0-aeb0-064288d8782c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:07.829024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.829979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.830443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.830956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.866862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:07.867521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:07.867712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:07.867987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:07.868753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:07.869647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.872124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.877639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 346/696] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:07.878494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:07.917170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:07.919002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:07.939943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.030563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a11d4b-b38e-4406-900d-ab9286817b83 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.034907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 351/697] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:08.036043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.064231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.080216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.080954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.081235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.081509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.156881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a9b18-826d-489f-9bd4-87151df960f4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.168812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.170031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.203246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.204329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.204788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.205364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.206261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:08.206769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.208509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.215651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 352/698] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:08.216830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.220357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.329542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.343876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d753-8740-4586-a7bd-8cf4a40e6f2b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.359682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 347/699] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 10:28:08.370777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.391815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.393050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.393533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.394013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.407266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.409173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:08.430043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:08.431066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:08.432751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:08.432751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:08.432751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:08.432751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:08.435272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 353/700] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:08.445805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.544149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d5aa-4a86-452e-b77a-6806b7765943 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.583683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.584518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.584804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.585083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.594222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.599230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def7bdf-d7ce-49a6-a4ef-dfa1d38d17cb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.600189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.601696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:08.626262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:08.627303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:08.628188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:08.628188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:08.628188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:08.628188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:08.628914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 348/701] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:08.634039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.665453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.667107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.668022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.668892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.670127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:08.670739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.672515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.684199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:08.692137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 354/702] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:08.701036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.774983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9176b-76a6-4dd1-9e89-7d30d1dd8f44 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.816978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.818023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.818666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.819150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.820183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:08.820885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.823395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.838986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34bcc-0f66-4d43-bbcd-1b075c0a50ed None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:08.845339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 349/703] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:08.851411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.882065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:08.883321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:08.883813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:08.884297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:08.893571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:08.895276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:08.913301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:08.914310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:08.915244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:08.918752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 355/704] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:08.921510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:08.933698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.027447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3705-c77f-4e5d-a2ac-c2c7e1b8a381 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.050279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.080489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.081763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.082287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.082846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.092092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.094658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:09.110638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:09.111611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:09.112568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:09.115133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 350/705] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:09.122309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.148810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf1ee0-c0b0-4c0c-a426-aea8fb5ffbc6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.219248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.219833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.220114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.220595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.223387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:09.223699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.225141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.255899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 356/706] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:09.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.313913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0368d7-0ceb-4dfd-a679-3cc64f637204 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.326521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.356003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.357095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.357477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.357878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.359397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:09.359779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.361281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcf52b-b076-4f78-b102-2e75727bee67 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.382576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 351/707] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:09.382576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645efec-2566-4757-a3a1-e1ff9d131629 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.382576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645efec-2566-4757-a3a1-e1ff9d131629 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.382576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645efec-2566-4757-a3a1-e1ff9d131629 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.386765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 352/708] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:09.390757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645efec-2566-4757-a3a1-e1ff9d131629 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.414049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2447-e200-4798-955f-50b2d6c056c7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.437652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.460129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.460883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.466639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.466639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.466639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:09.466639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.466639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935ad5-0de0-421b-8024-8bcef0d40c06 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.470835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 357/709] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:09.472033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c149ca6-b127-43cc-8428-66d53b3e5fce None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.472324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c149ca6-b127-43cc-8428-66d53b3e5fce None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.472602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c149ca6-b127-43cc-8428-66d53b3e5fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.497154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0c149ca6-b127-43cc-8428-66d53b3e5fce None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:09.499302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c149ca6-b127-43cc-8428-66d53b3e5fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:09.534211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645efec-2566-4757-a3a1-e1ff9d131629 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.581840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.582688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.582971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.583301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.584930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=7b4c6a5adbda4403883661e072aba906) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:09.585648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.588403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db702a-7f53-405c-9779-4d35e6b9873f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.602775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 358/710] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:09.605873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086a784-0cdd-4b0f-a8db-9ad4bf3c192f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.606968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086a784-0cdd-4b0f-a8db-9ad4bf3c192f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.607421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 353/711] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/role_assignments?user.id=7b4c6a5adbda4403883661e072aba906&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:09.608081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086a784-0cdd-4b0f-a8db-9ad4bf3c192f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.610912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 359/712] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:09.618374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.620061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.620730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.641919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:09.642845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086a784-0cdd-4b0f-a8db-9ad4bf3c192f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.643943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:09.690345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.762720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 354/713] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 10:28:09.815686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086a784-0cdd-4b0f-a8db-9ad4bf3c192f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:09.819178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.833813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:09.857674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:09.858441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:09.858753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:09.859028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:09.859751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:09.861206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:09.874875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:09.876178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:09.877004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:09.879882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 360/714] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:09.882474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:09.964808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69350d9a-1d62-4abf-b030-76d69e433a86 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:10.000428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0a47e-140e-4b29-a7d1-5db74f4e31a3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:10.025043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:10.026895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:10.027415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:10.027879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:10.038404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:10.039115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:10.039466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:10.039666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:10.040341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] RBAC: Adding query filter params (name=network) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:10.040648 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:10.043095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:10.044614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:10.046400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:10.052195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 361/715] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:10.054472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:10.073429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:10.074767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:10.075816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:10.075816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:10.075816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:10.075816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:10.078105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 355/716] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:10.084446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:10.101016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:10.111509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:10.193762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a9a73-870c-4b8e-b36a-397992ac424f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:10.236474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8358bc-73b2-4d6c-8715-cd7b3e42d002 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:10.239892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:10.240502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:10.240968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:10.241416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:10.242671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:10.243203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:10.245752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:10.256573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 362/717] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:10.278055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:10.279182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:10.279673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:10.280175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:10.281144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:10.281669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:10.283531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341721d-00fb-4462-bd7c-18c9369790db None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:10.298480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 356/718] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:10.310580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:10.485450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5642514-2461-42c4-821e-328840c467f0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:10.487860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:10.539218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:10.540252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:10.541284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:10.541736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:10.543056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:10.547081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd09bf8-29f2-4ee8-b5d6-9b284476fc1b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:10.662046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 363/719] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:28:10 2022] POST /identity/v3/users => generated 254 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:11.054915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57429-0011-40b4-801b-d22da31fa21a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:11.055972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57429-0011-40b4-801b-d22da31fa21a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:11.056557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57429-0011-40b4-801b-d22da31fa21a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:11.058809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 357/720] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:11.067215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f9c4-0c35-4417-9c46-6102e61495f8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:11.067968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f9c4-0c35-4417-9c46-6102e61495f8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:11.068479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f9c4-0c35-4417-9c46-6102e61495f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:11.088472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5aa0f9c4-0c35-4417-9c46-6102e61495f8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:11.090006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5aa0f9c4-0c35-4417-9c46-6102e61495f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:11.201576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 364/721] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:11.222719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef57429-0011-40b4-801b-d22da31fa21a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:11.309223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:11.334750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef57429-0011-40b4-801b-d22da31fa21a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:11.375228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:11.375932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:11.376219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:11.376495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:11.377237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:11.378658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8eccd-4c46-49c6-9bc3-dbcaa1c47f12 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:11.489178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 358/722] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:11 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:12.234119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a418f-186c-46bc-bb5c-57b7a62a0580 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.235189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a418f-186c-46bc-bb5c-57b7a62a0580 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.235668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a418f-186c-46bc-bb5c-57b7a62a0580 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.237287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 365/723] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:12.242946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc97b2-e620-4dac-9f95-a9349bc3cb58 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.243906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc97b2-e620-4dac-9f95-a9349bc3cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.244395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc97b2-e620-4dac-9f95-a9349bc3cb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.271577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-59cc97b2-e620-4dac-9f95-a9349bc3cb58 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:12.273145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-59cc97b2-e620-4dac-9f95-a9349bc3cb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:12.383794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 359/724] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:12.445853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a418f-186c-46bc-bb5c-57b7a62a0580 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:12.613437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:12.627394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a418f-186c-46bc-bb5c-57b7a62a0580 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:12.685474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:12.686287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.686705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.687473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.701200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01de-90ae-458c-b171-a183e4535d65 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.701896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01de-90ae-458c-b171-a183e4535d65 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.702406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01de-90ae-458c-b171-a183e4535d65 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.704447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 360/725] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:12.709786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:12.711754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.712372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.713268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:12.713415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:12.714480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:12.715360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 366/726] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:12.722952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:12.737698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:12.739371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:12.809732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:12.827995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 361/727] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:12.840642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591f310-e571-4abd-9d2a-ada98b925869 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:12.871977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:12.894578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:12.894578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:12.894578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:12.894578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:12.895225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:12.895225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:12.895225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:12.910575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 367/728] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:12.915542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:12.945374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.096174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd37eb7-3bf5-48f9-807c-101172f35d8f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.109157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b559-be72-4fd6-b8a5-0c62f7a184c2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.142864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.143953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.144237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.144510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.154573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.155402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.155772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.156053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.159353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.160821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.168491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:13.182911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:13.185049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:13.187205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b4180d0d-fb27-47ec-9868-4771292c84f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:13.188730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:13.190215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 368/729] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:13.192221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97e9e-f86b-4e3d-b5ae-eceaa6315fd3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.194580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97e9e-f86b-4e3d-b5ae-eceaa6315fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.194580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97e9e-f86b-4e3d-b5ae-eceaa6315fd3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.196824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 362/730] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:13.196824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.197687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 369/731] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:13.205453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97e9e-f86b-4e3d-b5ae-eceaa6315fd3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.266229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.279100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.322617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f97e78-e20e-4dd3-9042-59b5ed71260d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.323416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97e9e-f86b-4e3d-b5ae-eceaa6315fd3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.363715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.364292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.364583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.364860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.365595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:13.365904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.367820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0abc-e191-4226-b977-a0555b4dc40d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.375413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.377202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.377714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.378248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.379890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:13.380488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.382528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.383420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 363/732] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:13.385008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.385308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.385573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.399167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 370/733] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:13.400630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.410496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:13.412610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:13.507108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 364/734] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 10:28:13.508439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.520278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3462-1929-4047-91e3-174f8611d50e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.549765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.560217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.568271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.568271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.568271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.570483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.572522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.576787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:13.589299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:13.590607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:13.591797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:13.594167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 371/735] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:13.595557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.645316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1b929-df1d-4a37-a01d-ddbc5d188c34 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.685029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.686118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.686464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.686853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.698760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.700357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.716058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5d69-4219-4665-944f-715a496f15d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:13.721425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:13.722395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:13.723336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:13.723336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:13.723336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:13.723336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:13.727755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 365/736] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:13.730004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.761595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:13.763227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:13.763856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:13.764501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:13.765612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:13.767297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:13.872365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 372/737] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:13 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:13.873884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:13.951927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.960277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:13.971412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03745c01-b66c-4f6b-aa1a-39cc3f4669c0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.027863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6fe5-6721-440b-89ce-14016afe6b6f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.030527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.032621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.033197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.033954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.035005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:14.035464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.037138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c65f59-4656-41c0-8638-d84550b8c7a2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.063764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 366/738] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:14.065834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ddd5-96e4-4066-b714-0cf7236f21dd None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.066280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ddd5-96e4-4066-b714-0cf7236f21dd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.066727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ddd5-96e4-4066-b714-0cf7236f21dd None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.069925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 367/739] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:14.071055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30ddd5-96e4-4066-b714-0cf7236f21dd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.078470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.079331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.079736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.080102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.080932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:14.081334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.082934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424e3f6-d1e0-4148-8854-97c57c6c9410 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.097940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 373/740] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:14.099620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.100104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.100562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.118261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:14.120779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:14.205463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 374/741] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:14.206455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.216343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30ddd5-96e4-4066-b714-0cf7236f21dd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.244443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.278260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.279396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.279786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.280412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.295106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.297014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.308339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:14.320294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:14.323633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 368/742] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:14.325496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.332796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56a051-9ef4-408b-aab7-ab94f523840e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.370992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.372152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.372581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.372926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.381272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.383031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:14.400802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:14.403227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 375/743] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:14.405406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.452156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d67d9a-6501-435c-b7db-3206e73bd0e2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.494945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.494945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.494945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.494945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.508018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:14.516951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:14.517997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 369/744] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:14.523398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.528327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.537174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800110a-ebc1-40ad-b58b-c274275fd4e1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.603508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.612693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.613373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.613698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.613924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.615054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:14.615054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.616345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cf211-84b4-49eb-bfd4-6ac1bb4a6f87 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.629470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 376/745] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:14.635900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54d455-5fa4-4de6-981e-76695b12eec2 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.636211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54d455-5fa4-4de6-981e-76695b12eec2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.636478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54d455-5fa4-4de6-981e-76695b12eec2 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.641823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 377/746] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:14.641823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54d455-5fa4-4de6-981e-76695b12eec2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.647230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfd2ea-b40b-4400-8930-1f4a0e83d661 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.687085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.687876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.688189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.688551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.689484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:14.689820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.691499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.711843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 370/747] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:14.711843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.774207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54d455-5fa4-4de6-981e-76695b12eec2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.810631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.812017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.812461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.812777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.813501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:14.813810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.815219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdb123-3bf7-4c47-8b4e-e3d92244d171 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.825181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 378/748] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:14.826824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.827115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.827393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.840437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb217a-715f-46e0-b4b3-2d12cc6f1e08 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:14.844368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:14.845561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:14.856112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.861233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:14.879753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:14.880498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:14.880776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:14.881047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:14.888506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:14.889966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:14.903953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:14.904853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:14.905772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 371/749] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:14.906264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:14.922595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 379/750] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:14.922595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.026005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab1426-79ec-4e22-8186-06a06369e79d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.029894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abe95f-8fab-47c9-b167-570847133d2f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.069417 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.069673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.070156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.070267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.070444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.070565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.070741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.070835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.084233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.085870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.089904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:15.106524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:15.107818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:15.108715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:15.108715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:15.108715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:15.108715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:15.108715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 380/751] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:15.108985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.130572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.154129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.155635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.237301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030606-05f4-40fb-82eb-28ad5bf2ab1e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.277033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.278068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.278535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.279046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.280136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:15.280441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.281878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.290568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 372/752] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:15.290568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.302253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 381/753] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:15.302253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.367356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.388507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.410688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c85474-bf3c-4b96-b46a-619cae217185 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.418116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709f705-f48a-4d7a-a468-c1a9f8bbb211 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.450361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.451179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.451460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.451729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.452507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.453871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.455505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.456373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.456798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.457212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.458070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:15.458521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.460254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:15.469481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:15.470634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:15.471598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 373/754] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:15.472527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.482660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 382/755] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:15.482660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.601565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df4c8-e9aa-4f3c-b751-08fa3f8fa3e5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.613707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061c929-5a43-44db-b08f-ea8fc643a5c7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.621672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.646787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.648192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.648985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.649361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.652034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=23fabbe0e78d45f890b21fcf4b348721, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.653597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.653752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.654527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.654833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.655103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.667313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.668735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.670650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:15.687588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:15.689397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:15.691776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:15.691776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:15.691776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:15.691776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:15.691776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 374/756] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:15.693138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.718768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 383/757] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:15 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:28:15.718768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.850785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76b23-4f10-49f7-b2c0-b0de81e49b2c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.854961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1caed-dee4-400e-bc26-550442334f37 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:15.897102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.897783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.898067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.898345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.899191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:15.899592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.900982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.905083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:15.906514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:15.907019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:15.907471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:15.908324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 375/758] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:15.909200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.921146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:15.921146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:15.940715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:15.941764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 384/759] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:15.943385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:15.994259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:15.995313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.052321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559a8a-6a13-4cd0-b2c9-a3b6b958fa36 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.071493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd803ef-9893-401a-913f-0aa2414d5e0d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.091631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.092598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.092712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.093112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.094159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.094475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.096173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.114724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 376/760] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.117433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.117851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.120490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.120877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.121643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.122694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.123074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.125519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.131966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 385/761] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.133634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.252808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce94fe-6055-4153-a6ef-8df1349271c1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.265347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.268327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e77-ef12-4a0e-8239-b327196a803b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.272777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.293610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.295065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.295572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.295837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.296684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.296997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.298473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.310519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.311414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.311690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.311959 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.322646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 377/762] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.322646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.322935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.322935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-dd1fb5cd-d210-4258-98eb-a6a630a752b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:16.337972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:16.339238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:16.340243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 386/763] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:16.341441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3597e0-63e3-434a-a102-8e1c1a45875c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.341441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3597e0-63e3-434a-a102-8e1c1a45875c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.341441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3597e0-63e3-434a-a102-8e1c1a45875c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.342707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 387/764] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:16.347246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3597e0-63e3-434a-a102-8e1c1a45875c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.445018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254de097-e9b0-4192-9ed1-9cea277eb4cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.464941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3597e0-63e3-434a-a102-8e1c1a45875c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.487225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.487974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.488314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.488588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.489429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] RBAC: Adding query filter params (interface=public, service_id=639e14f51a0947fa947d5c34b4382140, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.489748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.491305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.500864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 378/765] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/endpoints?service_id=639e14f51a0947fa947d5c34b4382140&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.501859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.517301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.517959 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.518267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.518694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.551230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.551888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.602138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.604732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.635590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1e53-eab1-4f83-b597-88b33efa254a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.677946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.678787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.679074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.679348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.680125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.680428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.683315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078e91-193a-4ee2-adf7-4d0696fc12ee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.696883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 379/766] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.698218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.698619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.698942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.716226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:16.717407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:16.727149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 388/767] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:16.727982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.797119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.801960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 380/768] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:16.803093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.844559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:16.844906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658587-e5d5-4ec1-b1a2-bcf36973d584 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.887378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.888064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.888356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.888609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.889315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.890756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.906580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 389/769] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:16.906580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:16.939059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152bd38-66ec-4a51-bd84-7feb9a992991 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:16.981734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:16.982313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:16.982619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:16.982925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:16.983915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=7b4c6a5adbda4403883661e072aba906) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:16.984506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:16.986080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:16.999087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 381/770] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/role_assignments?user.id=7b4c6a5adbda4403883661e072aba906&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:17.000079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.057362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa2767-1f76-4dc0-9a6d-5530293d7234 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.106577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.119048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.119931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.120218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.121658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.134050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.136208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:17.157373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:17.158666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:17.159758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:17.159758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:17.159758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:17.159758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:17.159758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 390/771] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:17.169819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.196527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92ff95-3589-4b4d-bbbe-6a6a9d025f4e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.210986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.239120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.239952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.240246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.240523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] PATH_INFO: `/v3/services/network` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.241176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.242579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:17.258012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:17.259085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:17.261013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 382/772] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:16 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:17.266359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.302382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d21a69-b325-4f25-92a0-95d191edf74b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.363895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.364943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.365454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.366018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.366977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:17.367481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.369433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.396099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 391/773] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:17.403607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.418772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3dad3-c6be-47a1-aa81-69a2686cc2dc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.494741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.495614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.496578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.496895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.497202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.498043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] RBAC: Adding query filter params (name=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:17.498357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.499775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.513539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 383/774] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:17.520989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.587589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.618175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03f54-8a61-4c39-8bfe-01b92a49fe54 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.658792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.659990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.660559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.661058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.669093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.675191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.677291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f7b44-5463-4f23-84de-5e84a5c71c26 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:17.704730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:17.705690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:17.706614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:17.714582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 392/775] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:17.721449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.722131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.722418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.723747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.724542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:17.724856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.725096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.726342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.742567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 384/776] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:17.744436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.840820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.856964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c217-6066-4286-99d1-b12bd17128a8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:17.894381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:17.895077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:17.895368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:17.895648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:17.896443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] RBAC: Adding query filter params (interface=public, service_id=eaeb3d9dbac6480791868443376c9cde, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:17.896757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:17.902569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:17.905151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 385/777] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/endpoints?service_id=eaeb3d9dbac6480791868443376c9cde&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:17.910996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:17.926913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:17.934239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bda09a-ff73-4ea9-a29a-f9d569d63f06 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.003135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.003859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.004198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.004477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.005272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:18.005584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:18.007055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906ecc-0238-44dc-89f8-f21cd9a7e5f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:18.020898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 393/778] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:18.022223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd093d-cf57-42de-8029-5efae19ecf0e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.022513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd093d-cf57-42de-8029-5efae19ecf0e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.028643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd093d-cf57-42de-8029-5efae19ecf0e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.030269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 394/779] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:18.031047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd093d-cf57-42de-8029-5efae19ecf0e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:18.044447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c54b-0963-49ab-8a36-e20054e9c2a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.084276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.084983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.085359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.085643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.086328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:18.087818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bcb8c-3785-4d84-b8a7-eca09aa76665 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:18.098801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 386/780] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:17 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:18.100248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.100578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.100916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.137093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:18.138349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:18.163873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:18.185911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd093d-cf57-42de-8029-5efae19ecf0e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.280982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.282201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.282931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.283415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.284446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=46b53ca68e1a4445962f4604d1b90d9c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:18.284914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:18.286706 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:18.298681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 395/781] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:18 2022] GET /identity/v3/role_assignments?user.id=46b53ca68e1a4445962f4604d1b90d9c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:18.309771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 387/782] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:18.348495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:18.376466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:18.469392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be97ca-e0fd-44fb-b620-b292ebaee22a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.515624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.517171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.517621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.518043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.532222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:18.540369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:18.542514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:18.543551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:18.545946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 396/783] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:18.551860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:18.683913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08804b0f-aea3-4227-b025-d73e12901b4b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.741177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.742177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.742693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.743116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.744672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:18.745133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:18.747008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:18.763558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 388/784] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:18.768812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:18.862924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:18.878457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edefa98-88c2-4815-bd98-ab3f755eda33 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:18.920021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:18.920890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:18.921171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:18.921445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:18.938917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:18.940377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:18.960326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:18.961273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:18.962188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:18.962188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:18.962188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:18.962188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:18.962188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 397/785] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:18.967884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:18.982230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:19.102376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c2c4-6659-4737-be20-659164a45ff7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:19.153561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:19.154266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.154573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.154856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.155532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:19.155836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:19.157201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:19.171074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 389/786] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:19.176842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:19.331974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54797-6386-47b7-b14e-c52a65951fc9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:19.367555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:19.373671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:19.374861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.375351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.375823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.385549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:19.387819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:19.404601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:19.405594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:19.406533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:19.409396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 398/787] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:28:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:19.414952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:19.542123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:19.584980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb5b8-d82f-4c31-9193-188524171d55 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:19.632664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:19.633356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.633645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.633922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.635382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:19.635706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:19.637282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40b4d-491f-4a9f-b269-3842da5b97a3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:19.666493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 390/788] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:28:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:19.673643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:19.768584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6df5-7ec5-4839-a9cc-ef9b0eb28a2b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.768584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6df5-7ec5-4839-a9cc-ef9b0eb28a2b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.768990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6df5-7ec5-4839-a9cc-ef9b0eb28a2b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.770491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 391/789] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:19.776642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69078f8-d351-49ce-8b6e-d3b0d55fd37f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.776926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69078f8-d351-49ce-8b6e-d3b0d55fd37f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.777190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69078f8-d351-49ce-8b6e-d3b0d55fd37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.795344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e5207-97cf-4205-9103-8ca42d20f966 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:19.796540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f69078f8-d351-49ce-8b6e-d3b0d55fd37f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:19.798010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f69078f8-d351-49ce-8b6e-d3b0d55fd37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:19.843493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:19.844067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.844346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.844618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.901277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 392/790] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:19.903134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417a6-767d-49ef-bb64-348bf94b26ca None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.903544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417a6-767d-49ef-bb64-348bf94b26ca None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.903922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266417a6-767d-49ef-bb64-348bf94b26ca None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.905929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 393/791] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:19.912188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:19.912702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:19.913204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:19.932135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:19.934408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:19.947063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:19.959245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:19.961704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:19.994155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.015692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 394/792] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:20.016550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.110751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 399/793] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 10:28:20.115377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.148275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7862d-37d0-47be-a40a-3b1fd5c385bd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.190203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.193538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.193978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.194421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.195327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.197032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:20.210597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:20.211786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 10:28:20.212838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:20.220240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 395/794] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:20.221789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.225246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.235466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222098a-d13f-4421-898a-1836c14ddf7b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.274398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.275622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.275988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.276332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.281711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.293829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:20.308601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:20.309813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:20.310973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:20.312763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 400/795] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:20.313826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.347608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719a65-6192-4390-ae9c-abe17720f96d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.398403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.400749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.401468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.402119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.421893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.425262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.440485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be126-362b-469c-9d6e-98865779d257 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:20.446216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:20.448026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:20.449008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:20.449008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:20.449008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:20.449008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:20.451294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 396/796] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:20.452502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.478603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.479579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.486442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.486442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.486442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:20.486442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.486442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.494119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 401/797] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:20.494119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.512180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.577459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.581636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928633a-8c1a-4886-9cfe-066fc20df13b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.608147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0fce-7b02-4786-9415-30e0e78177a0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.623780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.625418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.625877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.626350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.627511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:20.628095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.629630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.640302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 397/798] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:20.641547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.646728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.647548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.647984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.648345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.649233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:20.649548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.651460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.669300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 402/799] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:20.670372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.757086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.780006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793494d0-e8d6-49f0-8100-df16dd331a54 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.784955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41e128-e1e1-4999-8249-f68419393a1e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:20.823451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.824447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.824908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.825377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.826249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:20.826501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:20.827280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:20.827614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.827717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:20.828050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:20.829784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.842691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 398/800] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:20.842691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.847896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:20.849476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:20.880565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:20.881522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:20.882618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:20.882618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:20.882618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:20.882618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:20.883271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 403/801] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:20.883634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:20.885123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:20.993411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a57aa-2679-4df3-90e4-77df6cc1d649 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.011237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f698f-a3f3-4d41-999f-95f5599828db None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.035131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.035928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.036208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.036479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.042771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.045301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.046776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.051636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.052330 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.052617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.053056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.053814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.058012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f97fd-eb40-4b4f-b603-7f20c8d99c74 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:21.062535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:21.063602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:21.064503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:21.068471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 399/802] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:21.069610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.168224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.179989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 404/803] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:21.182173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6618f4-45e3-4f81-8564-d0e9999b01eb None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.182395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6618f4-45e3-4f81-8564-d0e9999b01eb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.182740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6618f4-45e3-4f81-8564-d0e9999b01eb None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.183993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 405/804] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:21.184736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6618f4-45e3-4f81-8564-d0e9999b01eb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.185064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b7a1f-bbc9-41d2-b910-5efad7609faf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.238955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.239636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.239913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.240182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.240912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:21.241482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.242949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f765721-bb3b-4e28-b4b4-dfa97b53353b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.263047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 400/805] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:21.264929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a239-c1d2-417f-ab7e-7da9823385c6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.265243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a239-c1d2-417f-ab7e-7da9823385c6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.265524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a239-c1d2-417f-ab7e-7da9823385c6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.267242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 401/806] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:21.273230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.273599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.273796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.291624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:21.292829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:21.330709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6618f4-45e3-4f81-8564-d0e9999b01eb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.377638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 402/807] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:21.378570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.386866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.388366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.388861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.389380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.390433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:21.391068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.392774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f985ca-d86b-48bc-be33-0edacdff2337 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.408900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 406/808] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:21.410710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.411175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.411786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.429071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:21.430275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:21.464131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.479856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.504033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6b38c-0220-40d1-9689-8cdd0f4dd4b6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.514776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 407/809] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:21.516834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.545367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.550208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.550208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.550208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.554741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.556830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:21.571710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:21.574152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:21.575145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:21.578516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 403/810] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:21.579440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.635724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6f877-e73f-4447-8898-cc15d5680fa9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.671594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.672792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.673221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.673644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.674466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.675915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:21.688697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:21.689855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:21.690872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 408/811] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:21.691844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.704824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324e3f6-7dd7-43df-9920-5b990cb1c4b2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.744985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.753358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:21.758639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.761815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.761815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.761815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.761815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:21.761815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:21.762461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:21.779017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 404/812] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:21.779017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:21.863974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9b244-1d50-498f-86eb-ff79d358bdb7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.925601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:21.926924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:21.927392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:21.927846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:21.948196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:21.958273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e0438-ff36-42b1-bce4-590b25bd10b6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:21.963193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:21.964362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:21.965201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:21.965834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 409/813] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:21.966845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.015117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.015799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.016076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.016354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.017195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.017507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.019006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.053169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 405/814] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.053825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.126675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.132443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe7fac-dcf1-45e4-8fb0-e741801c2ba3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.151272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.188497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.189600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.190178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.190774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.191626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] RBAC: Adding query filter params (name=network) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.192045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.193579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.199646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 410/815] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.200700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.220042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c22bea-beee-4652-9234-4c4c0c29d7da None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.261461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.262161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.262476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.262872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.263734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.264050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.265438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.282879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 406/816] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.283898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.315484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95122-201e-4055-b54c-77f5f5b2d870 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.356933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.357846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.358203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.358627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.370978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.372804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:22.393254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:22.399557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 411/817] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:22.401856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.406581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66dac-18a6-45b0-9c98-f724834161de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.423452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.425252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.447746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.448620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.448910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.449191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.450095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.450666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.452565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.459054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 407/818] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.460049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.533842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46151-6d04-4f65-95df-acd2220856df None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.572336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.573195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.573553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.573912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.586213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.588642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.592340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bde94-bcba-42b9-bc50-8f3a7db62702 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:22.612590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:22.616088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:22.619369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:22.619369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:22.619369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:22.619369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:22.619369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 412/819] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:22.620104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.646077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.646741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.647029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.647311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.648049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.648355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.649939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.669296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 408/820] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.670469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.719592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.731227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.771900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a9d0-e21f-46df-b069-255c573679a2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.784613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6510cf-3c40-49a5-925c-f455ecef0a09 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.813699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.815351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.816108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.816857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.818183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.820192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.822886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:22.823582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:22.823867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:22.824145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:22.824880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:22.825189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:22.826615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:22.840459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 409/821] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:22.841135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:22.969027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:22.978591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01cb89-883b-47d4-81f8-a6bd13e3b93f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:22.979459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 413/822] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:22.982018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.075311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.076249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.076540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.076823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.096928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.098578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:23.126988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:23.128038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:23.128940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:23.144287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 410/823] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:23.163651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.223577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38a9bd-8be8-4dcc-b419-2a7a4ed5f7fc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:23.268017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:23.284397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.285237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.285516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.285785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.297680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.299324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:23.319756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:23.320805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:23.323667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 414/824] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:23.341377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.375465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c678c0-beed-456b-b4fd-1833f8d309d1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:23.429336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.430055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.430337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.430634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.434874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:23.435192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.436761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.450972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 411/825] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:23.457804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.479425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635997-fb00-456f-9e92-40dc0056bde6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:23.489328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:23.521408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.522081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.522358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.522821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.523621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:23.523928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.525777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572cad9-8444-4193-b1cf-5b3d368ca1f9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.547516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 415/826] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:23.548653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859391e-c66b-4797-82eb-dfdaa03d63f3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.548937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859391e-c66b-4797-82eb-dfdaa03d63f3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.549364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859391e-c66b-4797-82eb-dfdaa03d63f3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.550824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 416/827] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:23.555629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859391e-c66b-4797-82eb-dfdaa03d63f3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.588729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f06951-a6f7-4792-8d42-fcc1416a57fd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:23.612181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:23.629033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.630053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.630409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.630728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/7b4c6a5adbda4403883661e072aba906/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.632715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=7b4c6a5adbda4403883661e072aba906, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.635034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ae44-8b5b-4f7c-961f-eed905439298 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.679111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859391e-c66b-4797-82eb-dfdaa03d63f3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:23.711323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 412/828] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:23 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/7b4c6a5adbda4403883661e072aba906/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:28:23.714202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.714735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.715820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.732085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:23.732896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:23.733182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:23.733459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:23.749142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:23.750660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:23.752942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:23.754755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:23.778151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:23.779360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:23.780475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:23.780475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:23.780475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:23.780475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:23.786452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 417/829] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:23.787592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.867886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 413/830] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:23.893565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:23.902957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:23.926616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:23.941116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c8f79-94d8-4a8d-b829-ec3c9904870e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.002150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.003725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.004203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.004690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.005680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:24.006365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.009670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.011700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bc27-8fe6-4e3f-a671-53ded3775ce4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.023953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 418/831] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:24.031811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.057261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.058402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.058912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.059510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.060414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.062016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:24.075343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:24.076462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:24.077321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.080220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 414/832] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:24.086097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.169903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:24.184780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e763ec5-8f69-4519-9e0d-b90f1a6a63b3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.217051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691ae80-32c1-4a5a-9cb5-5497ff09ccb1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.246501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.248368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.248820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.249268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.259316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.260267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.260729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.261108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.261938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:24.262385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.263391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.264004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969ec4f-840f-4f33-8a9b-50a40414254f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.265556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.274467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 415/833] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:24.276686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae689f-7cb7-43c5-9b09-c7992d7c65fb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.276998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae689f-7cb7-43c5-9b09-c7992d7c65fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.277250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae689f-7cb7-43c5-9b09-c7992d7c65fb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.279048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 416/834] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:24.280989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae689f-7cb7-43c5-9b09-c7992d7c65fb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-70c5fa7d-bf6e-4fc9-a0d1-9a9e7c812169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:24.292325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:24.294011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 419/835] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.295701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.296956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.318001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:24.319245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:24.328791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:24.397241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 420/836] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:24.397818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.435172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae689f-7cb7-43c5-9b09-c7992d7c65fb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.472676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.479246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:24.482262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 417/837] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:24.483231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.529535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb132-2b20-490d-a20a-1c59259adff5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.582039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.583170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.583864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.584450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.585487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:24.586172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.589223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.607984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 421/838] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:24.616395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.617812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:24.633686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372264e-3e14-4547-9a2d-142e900ce5bf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.676845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.678245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.678529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.678833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.679929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.681441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:24.697214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:24.698346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:24.699225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:24.700008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 418/839] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:24.705205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.745553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719b678-b936-4b2a-a2fe-3d2d0a2165bc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.783769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:24.789725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.790807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.791137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.791523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/46b53ca68e1a4445962f4604d1b90d9c/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.813836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=46b53ca68e1a4445962f4604d1b90d9c, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.815603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.835672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c428f-31a9-41a5-a63c-449952f5cb50 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:24.880185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:24.881257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:24.881766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:24.882283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:24.882743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 422/840] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:24 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/46b53ca68e1a4445962f4604d1b90d9c/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:28:24.883441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:24.883944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:24.886588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cc0b5-f38c-4330-b285-90c4ecef3316 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:24.897657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 419/841] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:24.903721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:24.976278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:25.030927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ab4b-76e4-4ad2-96ff-0da6d42b0120 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:25.073340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:25.074026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:25.074310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:25.074664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:25.075350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:25.075663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:25.077058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310a4b4-b0bb-41ea-897b-04ef45fe773a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:25.086413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 423/842] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:24 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:26.022268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ca144-0626-4f66-bb76-73f65c93167e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.023356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ca144-0626-4f66-bb76-73f65c93167e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.023968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ca144-0626-4f66-bb76-73f65c93167e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.026208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 420/843] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:26.033795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.035322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.036045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.062139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:26.064613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:26.151510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 424/844] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:26.176748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ca144-0626-4f66-bb76-73f65c93167e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:26.306161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ca144-0626-4f66-bb76-73f65c93167e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:26.344793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:26.356689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:26.357512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.357801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.358080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.370116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:26.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:26.379184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:26.380137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 421/845] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:26.389575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:26.454770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:26.526843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065ce7-9968-4f03-85cf-6ad22fe0de0e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:26.566279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:26.567574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.568086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.568582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.569553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:26.570080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:26.572269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c2fc7-4b77-4fa7-a8ef-79e22953510d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:26.582383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 425/846] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:26.591665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:26.698269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9a0e-b076-4132-b985-91ae98dbf029 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.698673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9a0e-b076-4132-b985-91ae98dbf029 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.698855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9a0e-b076-4132-b985-91ae98dbf029 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.700814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 426/847] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:26.707726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.708368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.708856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.725292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d767a8e-460c-4f58-90e4-2d1e025275d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:26.730712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:26.732193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:26.767592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:26.768383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.768659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.768928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.781742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:26.783216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:26.803042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:26.803995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:26.804896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:26.804896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:26.804896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:26.804896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:26.804896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 422/848] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:26.809888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 427/849] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:26.811244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:26.834583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:26.868986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:26.877719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:26.942345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3b2b-53fd-4608-b9f2-1e6fb70556f0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:26.945376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111092-c2a6-492e-9019-d16a91e1ae00 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:26.980098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:26.980881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.981168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.981446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.982242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:26.983756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:26.987175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:26.988265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:26.988610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:26.988977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:26.989770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:26.990132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:26.991593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:27.010036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 423/850] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:27.011120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.047274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 428/851] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:26 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:28:27.144820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:27.151884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf65e-b3f2-44f3-bffc-764d0fd001cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:27.208405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:27.209219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.209502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.209824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.219134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:27.220666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-71fbdc06-4193-44de-a9fc-1cde6e868614 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:27.238033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:27.239775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:27.241508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 424/852] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:27.248427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.364619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5b35-893b-4dba-ab6d-a386820d2bab None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.365266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5b35-893b-4dba-ab6d-a386820d2bab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.365812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5b35-893b-4dba-ab6d-a386820d2bab None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.367827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 425/853] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:27.375937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f3a5-98ac-4b96-b627-3a49d864dbaf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.376501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f3a5-98ac-4b96-b627-3a49d864dbaf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.376779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f3a5-98ac-4b96-b627-3a49d864dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.395352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:27.396940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8262f3a5-98ac-4b96-b627-3a49d864dbaf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:27.398275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8262f3a5-98ac-4b96-b627-3a49d864dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:27.402504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6196-484c-4adf-b5b4-49cb67931b20 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:27.448760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:27.449462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.449746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.450264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.451690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:27.452065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:27.453840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:27.482296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 426/854] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:27.484601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4b848-83fb-45c2-bfaa-f245aa8b9ccd None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.484948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 429/855] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 21 10:28:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:27.485310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4b848-83fb-45c2-bfaa-f245aa8b9ccd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.485553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4b848-83fb-45c2-bfaa-f245aa8b9ccd None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.493302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.498593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 427/856] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:27.498593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.498593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.499074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.522350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:27.523598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:27.611301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 428/857] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:27.615989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.636554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729205e-9039-42f8-af3c-959c3eabd4a0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:27.642328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:27.680308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:27.681056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.681916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.682206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.713139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:27.740033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd02c3-5bd8-4475-95a7-9541dab6ea77 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:27.765354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:27.766957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:27.784905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:27.785576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:27.785957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:27.786157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:27.787328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:27.788266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:27.886654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 430/858] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:27 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 10:28:27.887600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.898916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 429/859] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:27 2022] POST /identity/v3/services => generated 269 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:27.900658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:27.986130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:27.992335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.035452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01ce5-510d-4600-888b-dc2dbe4cbbbd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.036013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439155f1-f241-4f75-83a5-aa1f450eee02 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.078079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.079816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.080365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.080842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.086043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.086888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.087730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.088171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.096133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:28.101035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.102501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:28.105653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:28.106621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:28:28.107488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:28.109917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 431/860] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:28:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:28.115816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:28.124331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:28.125221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:28.126090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:28.126090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:28.126090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:28.126090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:28.126090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 430/861] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:28.134989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.261712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413df9-9861-4cad-8d2f-d2d1d472571c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.279116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fdc9d-d452-4551-afa9-b7000c0d4593 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.279759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.301105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.301798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.302078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.302346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.303244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:28.304237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.306105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.307495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.319883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 432/862] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:28.325756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.327342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.328025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.328309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.328584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.335109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:28.335109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.335109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.349378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 431/863] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:28.352036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.466341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aafaf-47d9-4d4b-8c6f-f53ce89776cb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.466756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5ab17-2d5e-4a70-b890-9b39d618a12f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.507931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.509773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.510592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.510955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.515532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.516579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.517038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.517498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.523481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.525363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.527959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.530016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:28.548252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:28.549202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:28.550115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:28.552777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 432/864] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:28.563662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:28.564584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:28.565582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:28.565582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:28.565582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:28.565582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:28.566294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 433/865] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:28.567588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.576232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.628923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.656159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.714314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3a2a-152d-41a4-9e0b-56b2b2353d71 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.759716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f15e1e-b494-4426-a10c-f695603175db None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.760521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.761635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.762482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.763012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.765034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:28.765627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.767321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.782616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 433/866] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:28.792400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.819315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.820343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.820788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.821232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.823021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:28.823514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.825948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:28.850645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 434/867] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:28.852892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:28.929574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c6d72-695b-431d-8f49-50f0faba6f9a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:28.932961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.988099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:28.989099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:28.989538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:28.990370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:28.991391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=7b4c6a5adbda4403883661e072aba906) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:28.991915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:28.994380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:28.995637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.006307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90878e5-ed99-4e7a-8806-516cfc62f108 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:29.011919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 434/868] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/role_assignments?user.id=7b4c6a5adbda4403883661e072aba906&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:29.048223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:29.049450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.049942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.050419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.060348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:29.062215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:29.080383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:29.081996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:29.082976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:29.085837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 435/869] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:29.092459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:29.235867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d426791-3a8a-4082-bb29-59f20699262d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:29.282094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:29.282912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.283098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.283373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.284224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:29.284534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:29.286268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5293df-4591-487f-9ab8-fae945c04869 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.318505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 435/870] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:29.327308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:29.387593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530e33d-a239-4a02-ba42-b51f3c10cc70 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.387947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530e33d-a239-4a02-ba42-b51f3c10cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.388167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530e33d-a239-4a02-ba42-b51f3c10cc70 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.389804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 436/871] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:29.395166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.395456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.395736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.420791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:29.422353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:29.451118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:29.475942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356c1d-363e-4db9-b2ed-243fe01039c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:29.526382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:29.527187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.527471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.527862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.538660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 437/872] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:29.557184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:29.567807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:29.655594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:29.657206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.704881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09883b02-3176-4dcd-b653-212ce8c64e95 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:29.751690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:29.752981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.753477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.753946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.755223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:29.756482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:29.771433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:29.772653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:29.773880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 438/873] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:29.782145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:29.808185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:29.818649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 436/874] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:29.819552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:29.919009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c70e5e-e172-4304-8446-0ace31a07dbc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:29.963548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:29.964679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:29.965186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:29.965672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:29.966664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:29.967191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:29.968893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:29.975255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 439/875] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:29.981342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.044166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.085108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.109159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb2dd-daaa-4644-b513-c738d3193894 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.113329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5c281-4212-4392-a1eb-b394a2de7455 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.169374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.170004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.170289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.170625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.171371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:30.171676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.173065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb8c10-2079-4915-818a-286bc669fbe8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.173550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.175089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.175691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.175969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.182722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 440/876] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:30.184579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3b62-a0c1-4c50-a467-22c5ab2f5594 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.184925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3b62-a0c1-4c50-a467-22c5ab2f5594 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.185268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3b62-a0c1-4c50-a467-22c5ab2f5594 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.187037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 441/877] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:30.194816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.195030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef3b62-a0c1-4c50-a467-22c5ab2f5594 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.196552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1c31298d-c7c6-4ae3-9c3b-2e48585d5b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:30.216474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:30.217434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:30.218321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:30.218321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:30.218321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:30.218321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:30.218321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 437/878] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:30.219745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.219745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.219745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.237718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:30.238990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:30.329361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef3b62-a0c1-4c50-a467-22c5ab2f5594 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.331131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 438/879] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:30.332804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.381046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.381722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.382285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.382581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.383454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] RBAC: Adding query filter params (interface=public, service_id=9608c66ab4ec4beabd0c31ef7ffb8a9f, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:30.383798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.385832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.386355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.394619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 442/880] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/endpoints?service_id=9608c66ab4ec4beabd0c31ef7ffb8a9f&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:30.395867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.416621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.488126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f1ad-b717-4c19-8ca2-71ab7e7eee46 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.523655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e4d0-3ea5-417b-99df-cf53a2fbf113 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.535863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.536957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.537369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.537924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.539199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:30.539691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.542450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.569408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 439/881] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:30.569408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.569687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.570198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.570473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.570773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.571421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.574095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:30.589373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:30.590397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:30.591617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 443/882] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:30.592462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.685782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.687675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:30.696171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e67b42-1029-40c7-b33e-7417563bf583 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.715794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4ecc-f010-4e61-81bb-c07d5236719d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.749061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.750187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.750769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.751305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.752554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.754948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.756279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.757926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.758391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.758813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.764870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 440/883] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:28:30.766076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.770223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.772769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:30.798465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:30.800109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:30.801532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:30.805842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 444/884] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:30.813881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:30.908570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd348c41-eae1-49ff-bd45-ce3f6378ab7d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.957977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f208c-4bc3-4e34-914f-e8fc91a0ceda None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:30.960902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:30.963088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:30.963371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:30.963690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:30.964421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:30.964759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:30.966168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:30.972882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 441/885] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:30.979686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.007351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:31.012702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.014097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.014751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.015551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.035683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:31.035683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.035683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.035683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:31.045564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 445/886] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:31.052354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.109686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451316b5-67ff-4c5a-b8a9-71af7438f5f1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.158167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.158860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.159144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.159609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.160516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:31.161135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.162843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.171545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 442/887] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:31.206059 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.226585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9be6-6027-49e5-9487-ef0b9ccbb954 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.290784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.291807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.292317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.292806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.293859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=46b53ca68e1a4445962f4604d1b90d9c) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:31.295373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.299659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1ad4-ef8a-4d99-a404-b2779a6ff669 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.305271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:31.318585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 446/888] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/role_assignments?user.id=46b53ca68e1a4445962f4604d1b90d9c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:31.328680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2d10-b5cb-4231-98f6-dc6bbc2516e6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.329013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2d10-b5cb-4231-98f6-dc6bbc2516e6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.329252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2d10-b5cb-4231-98f6-dc6bbc2516e6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.333527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9df19-dffd-4320-9418-1c87a30967f6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.335325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 447/889] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:31.336794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.337204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.337645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.358161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:31.360795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:31.384004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.384988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.385337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.385608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.386434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] RBAC: Adding query filter params (interface=public, service_id=3511802bae8649b3ab8ab2d0f01dd3ed, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:31.386776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.388188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.401489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 443/890] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/endpoints?service_id=3511802bae8649b3ab8ab2d0f01dd3ed&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:31.402837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.454128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:31.475190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 448/891] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:31.504485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.545017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189be9ac-6606-4e17-9982-c2da8680afc0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.590292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.591048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.591332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.591599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.592298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.593812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.603629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 444/892] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/services => generated 1268 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:28:31.644920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9321a85-7c88-4921-bd0c-067eb4864cbe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.689338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.690169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.690448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.690753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.691410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.692818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:31.706380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:31.707532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:31.708371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 449/893] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:31.714647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.733467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:31.842463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6df51-8459-4a0a-b05d-bb31a0d4e105 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:31.880004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:31.880929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:31.881365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:31.881793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:31.882716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:31.883176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:31.884807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ec557-2967-44e2-988c-d4e342097192 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:31.894650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 445/894] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:31.897320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:31.957497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:32.009154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028fc48-fcd3-4d4b-abb0-feb19f07a81b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:32.046247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:32.046966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:32.047249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:32.047526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:32.048241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:32.048551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:32.050004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f356f8-2a2c-4500-a137-6180d05d01ef None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:32.062761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 450/895] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:33.051345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f91-c5a4-4cc0-8cd3-ba438014a223 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.051923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f91-c5a4-4cc0-8cd3-ba438014a223 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.051923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f91-c5a4-4cc0-8cd3-ba438014a223 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.053643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 446/896] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:33.059919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.060660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.060660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.079696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:33.080917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:33.167271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 451/897] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:33.196442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3f91-c5a4-4cc0-8cd3-ba438014a223 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:33.300455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:33.329616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3f91-c5a4-4cc0-8cd3-ba438014a223 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:33.369096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:33.369906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.370191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.370467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.371481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:33.372868 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:33.385841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:33.387613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:33.389088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 447/898] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:33.400327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:33.460935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:33.533983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb642f-80ba-4496-acfa-08062e2d1de9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:33.578364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:33.579494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.580037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.580516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.581456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:33.581973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:33.583626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:33.590113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 452/899] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:33.596259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:33.729018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b9b1-601f-4586-b813-8cd7d87ff6ef None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:33.768749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:33.769438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.769733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.770006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.770767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:33.771081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:33.772492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc8d5-1140-4919-8139-b03d83ec6fcd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:33.778978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 448/900] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 21 10:28:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:33.784721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:33.887352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9c38-03cf-42b4-bc89-987a44aa7958 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.887559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9c38-03cf-42b4-bc89-987a44aa7958 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.887817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9c38-03cf-42b4-bc89-987a44aa7958 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.889689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 449/901] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:33.895422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.895795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.896073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.905176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:33.908916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a0871-c46a-4f5f-9129-72b04383aca3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:33.921927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:33.923616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:33.924477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:33.967541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:33.968535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:33.968822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:33.969098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:33.970422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:33.972496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b323c7-7d0b-475b-8bc4-716ba6f87628 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.018443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 450/902] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:34.043403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.070341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 453/903] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:28:34.073480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f879d8-833c-4eb7-8cde-f60b7e12aed5 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.074320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f879d8-833c-4eb7-8cde-f60b7e12aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.074837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f879d8-833c-4eb7-8cde-f60b7e12aed5 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.079819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 454/904] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:34.085065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.085587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.086259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.113832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:34.115555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:34.174818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3daf3-3e23-4c09-8ef7-af4c4afd63a0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.234683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 455/905] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:34.235011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.236116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.236430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.236711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.244963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:34.250992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:34.258884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:34.261774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:34.263496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:34.264993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:34.268313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 451/906] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:34.272122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.400267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:34.406871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c105e-b76f-4f29-8266-995146f1656a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.409770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6cb13-38af-470a-b767-21c14fdd035f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.448601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.449253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.449813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.450103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.450373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.450942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:34.451245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:34.451446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.451802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.452149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.452707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.452963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:34.454441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.463826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 452/907] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:34.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:34.469938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:34.470816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 456/908] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:34.472997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.476657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.551871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:34.614594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d067-a491-40da-9411-a9a713554f56 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.619425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5768bcf-2a48-4b02-b68e-0b335e5a9264 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.660275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.660498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.660966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.661245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.661343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.661519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.661626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.661853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.662243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:34.662573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:34.664004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e7812-3a9e-4206-9979-2d1fa7d6c9b1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.670752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 457/909] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:34.671977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132979c-dff6-4038-b6d7-3fd3658e4c78 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.672264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132979c-dff6-4038-b6d7-3fd3658e4c78 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.672538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132979c-dff6-4038-b6d7-3fd3658e4c78 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.674126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 458/910] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:34.678629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132979c-dff6-4038-b6d7-3fd3658e4c78 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.682382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:34.684341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.705062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-787e870b-a2f1-444f-9e91-896410efa33c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:34.708825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:34.709950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:34.710956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:34.710956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:34.710956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:34.710956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:34.711896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 453/911] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:34.713646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.713930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.714210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.736053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:34.737455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:34.814032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132979c-dff6-4038-b6d7-3fd3658e4c78 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:34.819190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 454/912] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:34.830949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.882424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:34.887335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:34.888015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:34.888295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:34.888565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:34.889283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:34.889583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:34.891073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:34.897639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 459/913] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:34.897639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:34.957496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9562cd-9acc-4551-986a-5f1aa9dd92bd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.005156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.005814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.006092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.006362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.007107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:35.007406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.008801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.012356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:35.025790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 455/914] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:35.029970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:35.034101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc533-5dad-4b8d-b67e-6c25879e8285 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.074459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.075208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.075561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.075806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.076518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.077885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.151673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:35.157086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38abbea-8a10-438d-ace5-e01e99bac4d2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.176018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 460/915] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:34 2022] POST /identity/v3/services => generated 241 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:35.176942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:35.221793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.223278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.223775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.224352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.225568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.227182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.356231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 456/916] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:35.370578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e4ac-0cda-4fb7-9ce2-fe71f8bef442 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.434879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.437276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.437609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.437861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.450627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.451054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.456315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:35.497176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:35.498231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:35.499275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 461/917] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:35.499872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:35.617225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a298a-0f13-4483-ba4d-dc1df3f02a33 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.659484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.660557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.661058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.661537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.662621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:35.663163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.664840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.687511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 457/918] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:35.693705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:35.822798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a76104-deac-488f-886e-6712ec2d93bb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:35.863989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:35.864658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:35.864937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:35.865206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:35.951271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:35.953264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:35.955337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:36.070450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 462/919] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:35 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:36.076960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.090598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:36.195467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942a637-1e07-4e76-8d34-b2733a63696a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:36.236780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:36.237599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.237884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.238161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.251120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:36.252641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c66ba078-cfe5-4901-8608-43542e8678ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:36.273307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:36.274279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:36.275772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:36.275772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:36.275772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:36.275772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.275772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 458/920] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:36.283725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.395965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:36.399035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60910bbe-7f8d-461d-8bae-ad39f10af804 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:36.427964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e67f2-b420-4e82-9016-8badb1ab78e8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.428628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e67f2-b420-4e82-9016-8badb1ab78e8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.429168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e67f2-b420-4e82-9016-8badb1ab78e8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.432019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 459/921] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:36.438306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.438479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:36.439118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.439226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.439449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.439721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.440092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.440437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:36.440739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:36.442142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:36.456305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 463/922] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:36.462051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.462785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:36.464289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:36.539207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:36.552619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 460/923] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:36.580376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.624351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa630d-5e16-4b9d-9b68-036edc952919 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:36.666639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:36.667476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.667770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.668049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.676257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:36.677759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:36.694314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:36.695334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:36.696236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 464/924] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:36.701991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.730297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9b11-5cca-44a1-8cb5-6548825dd164 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:36.731944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:36.786068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:36.787680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.788482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.788978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.789932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:36.793353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:36.795469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING keystone.server.flask.common [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 10:28:36.832683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74b13-3fd5-4997-ab90-cc068b23a4e0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:36.853580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.sql.core [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 10:28:36.853580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 10:28:36.853580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [parameters: {'id': '60e38c0d0d504ceaa4c5431587e32678', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 10:28:36.853580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60e38c0d0d504ceaa4c5431587e32678.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60e38c0d0d504ceaa4c5431587e32678. Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:28:36.861198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 10:28:36.862126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application yield resource Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise self.value Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application session.commit() Jul 21 10:28:36.863304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.flush() Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application compat.raise_( Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 10:28:36.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 10:28:36.866063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:28:36.867443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [parameters: {'id': '60e38c0d0d504ceaa4c5431587e32678', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.870398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:36.872762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60e38c0d0d504ceaa4c5431587e32678. Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 461/925] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Jul 21 10:28:36 2022] POST /identity/v3/projects => generated 262 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:36.876672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.896016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:36.897868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:36.898679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:36.899203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:36.900857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:36.901856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:36.904101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:36.922855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 465/926] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:36.929851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:36.933862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:37.011867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e9b02-1d1a-49a1-9e55-63f7bba65352 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:37.057560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:37.062808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:37.064316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.064819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.065295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.075048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:37.077867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:37.089125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52870-1706-4048-b216-4fe67d2e15c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:37.097319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:37.098325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:37.099243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:37.106792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 462/927] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:37.108350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:37.162373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:37.162373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.162373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.162373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.171197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] RBAC: Adding query filter params (role.id=41fa8686eed24d9f9d00303d8f67e01d, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=46b53ca68e1a4445962f4604d1b90d9c) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:37.171971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:37.176992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d117-8cf3-4245-a6d9-932570c7a7aa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:37.198620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 466/928] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:36 2022] GET /identity/v3/role_assignments?user.id=46b53ca68e1a4445962f4604d1b90d9c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=41fa8686eed24d9f9d00303d8f67e01d => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:37.354713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db8207-696c-4747-91d6-cf1665e663e7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:37.372240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:37.420802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:37.421823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.422260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.422752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.423998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:37.424485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:37.426888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c3d60-def4-4598-b4ca-df27be991b92 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:37.462083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 463/929] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:37.579866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a67e-2920-4eae-bc6c-3f4c076d5b74 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.580387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a67e-2920-4eae-bc6c-3f4c076d5b74 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.580387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a67e-2920-4eae-bc6c-3f4c076d5b74 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.582118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 467/930] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:37.589447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee7ee-73c3-4834-b3af-ede64e1ab735 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.589759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee7ee-73c3-4834-b3af-ede64e1ab735 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.590015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee7ee-73c3-4834-b3af-ede64e1ab735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.610891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9c9ee7ee-73c3-4834-b3af-ede64e1ab735 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:37.612436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9c9ee7ee-73c3-4834-b3af-ede64e1ab735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:37.689108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 464/931] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:37.716392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037a67e-2920-4eae-bc6c-3f4c076d5b74 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:37.800809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563439c6-c27c-4766-bb88-35e0bc44d7ac None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.801213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563439c6-c27c-4766-bb88-35e0bc44d7ac None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.801372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563439c6-c27c-4766-bb88-35e0bc44d7ac None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.803095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 465/932] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:37.810141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.810422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.810733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.831918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:37.833523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:37.851268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037a67e-2920-4eae-bc6c-3f4c076d5b74 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:37.892695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:37.893510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:37.893794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:37.894072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:37.894759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:37.896142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:37.920096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:37.921963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:37.923630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 468/933] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:37.927619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:37.929674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 466/934] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:37.946025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:37.952811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:37.992469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:38.062102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ac22-90b0-461f-a32f-b97e51cdbce4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.101128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4bd70-bd29-4dbd-b00b-69b72c263de3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.107159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.107972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.108356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.108732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.109572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:38.110089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.111923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.117525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 469/935] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:38.126632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:38.150986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.151958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.152355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.152695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.153644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.155042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:38.169396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:38.170662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:38.171715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:38.173177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 467/936] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:38.179485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:38.239569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:38.284393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53338fd-1eb0-458c-8340-cc1c7832a313 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.296998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:38.308352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a64c-c19f-4653-90c1-288ebd848fbc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.333639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.335537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.336007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.336436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.338408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:38.339353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.342130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.353623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 470/937] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:38.360319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:38.363297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.364334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.364772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.365228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.366411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:38.366947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.369885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.382459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 468/938] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:38.384619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:38.498648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019f55-29f7-4c73-b7d7-94d4d68ce428 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.512437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d4054-45db-4299-86f0-97ce85a79672 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.547299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.547879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.548186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.548437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.549222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] RBAC: Adding query filter params (interface=public, service_id=a239b8a804b944a187b2add6075680e8, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:38.549532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.551003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.552332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.553309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.553794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.553898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:38.554767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.556347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:38.556974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.558365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 471/939] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:38 2022] GET /identity/v3/endpoints?service_id=a239b8a804b944a187b2add6075680e8&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:38.558976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd07c-9e88-4754-8477-b74981a52acd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.566873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 469/940] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:38.567664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:38.694745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e8e4-ea5d-45b8-afc4-f86f27324bf4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:38.739590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:38.740328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:38.740776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:38.741220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:38.742207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:38.744290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167ef89-56a1-4d34-a2b7-57eac2964a60 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:38.751560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 472/941] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:38 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:28:39.434922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd83aa-5d77-43ec-a3aa-7042eab14dc9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.435923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd83aa-5d77-43ec-a3aa-7042eab14dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.436404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd83aa-5d77-43ec-a3aa-7042eab14dc9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.438310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 470/942] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:39.445594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.446174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.447267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.464720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:39.466254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:39.539124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:39.548466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 473/943] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:39.572880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd83aa-5d77-43ec-a3aa-7042eab14dc9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:39.594494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:39.700222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd83aa-5d77-43ec-a3aa-7042eab14dc9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:39.737329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:39.738506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.739043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.739543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.752200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e34049a9-7398-4e7b-9c55-a42a60d2bf69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:39.761319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:39.762304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:39.763167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:39.766058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 471/944] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:39.772539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:39.847078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b2c6-7ba9-4862-ad79-5b3c685fa4ba None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.847701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b2c6-7ba9-4862-ad79-5b3c685fa4ba None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.848176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7b2c6-7ba9-4862-ad79-5b3c685fa4ba None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.850034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 472/945] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:39.855492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.855776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.856050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.873666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:39.874903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:39.885725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f9f3c-ae7f-4d11-bb00-edd06443c70f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:39.923709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:39.924387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:39.924675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:39.924956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:39.925702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:39.926010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:39.927420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:39.942882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 474/946] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:28:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:39.944481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:39.952562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 473/947] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:39.978486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:39.989103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.011641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.060335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007b59-acaf-48ae-96fa-b308360950aa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.106577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.106577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.106577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.107073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.111026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020160-d793-4f6b-8be0-610cc4f45c13 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.123606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.125162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:40.143996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:40.145140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:40.146422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:40.146422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:40.146422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:40.146422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:40.152164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 475/948] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:40.153114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.167074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.167951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.168395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.168820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.169797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.171516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.224287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.sql.core [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 10:28:40.224287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 10:28:40.224287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [parameters: {'user_id': 'e0b877f4d83b428b8c0fe794a1f1b2ef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 10:28:40.224287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:28:40.235908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:28:40.236863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application yield resource Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise self.value Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application session.commit() Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 10:28:40.237802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.flush() Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application compat.raise_( Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 10:28:40.238885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 10:28:40.239847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 10:28:40.240673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e0b877f4d83b428b8c0fe794a1f1b2ef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:40.241517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 10:28:40.242476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 10:28:40.243524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.245822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 474/949] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 21 10:28:39 2022] POST /identity/v3/users => generated 160 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:40.252444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.289592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c3b1-fd9d-4306-8c92-c5c0c2cbd51e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.302307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.342284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.343490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.344051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.344256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.345108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.346503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:40.347157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.349410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.375880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 476/950] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:40.377029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.377604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45177e25-0b15-45a3-8786-805ef13ebaab None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.416232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.417186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.417667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.417972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.427224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.429053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:40.444881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:40.446948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 10:28:40.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:40.450239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 475/951] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:40.455180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.524491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d80f5-a6a9-45a3-9e3a-c5b02333831a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.575949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c5a93-5608-4009-9a32-f3e437653fe0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.578008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.579998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.580436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.580858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.591225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.593669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.608840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.615142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.615989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.616499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:40.617031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:40.618281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:40.626400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:40.627117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.627117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:40.627117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.627117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.627707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 477/952] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:40.628808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.664037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:40.682940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 476/953] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:40.760271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f34139-a0ab-4e76-9e73-8075b99fdb0b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:40.800747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:40.802111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.802563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.803005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.804025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:40.804483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:40.806899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5b353-9492-48aa-b8a8-ebb07e83efd3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:40.828740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 478/954] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:40.834624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:40.883433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c47f-c22f-4552-87b0-b404a7513b3f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.884786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c47f-c22f-4552-87b0-b404a7513b3f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.885024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c47f-c22f-4552-87b0-b404a7513b3f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.888364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 479/955] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:40.895756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:40.896244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:40.896686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:40.922221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:40.924131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:40.959909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6e43b-5925-4193-a4f6-9df25b3243bc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.004734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.005548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.005830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.006106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.018360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.019908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.023162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 480/956] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:41.043346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:41.044521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:41.045430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:41.045430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:41.045430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:41.045430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:41.047808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.049629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 477/957] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:41.050904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.068167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.103426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.168553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589e5d3-3a57-42d6-bb5e-fcbe209cb1a8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.180279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2df3f-0073-49e9-9560-c3a08c3f7978 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.205087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.205758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.206041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.206320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.207075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.208445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21cce1-83b3-4721-ab9f-c8bd8c2f17ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.215410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.216074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.216352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.216624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.217348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:41.219677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.219677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e88ea-6de3-4a7e-8f08-2ee755986605 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.232341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 478/958] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:41.233902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0e6be-1a3d-42da-bc03-081370eb88f4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.234232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0e6be-1a3d-42da-bc03-081370eb88f4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.234499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0e6be-1a3d-42da-bc03-081370eb88f4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.235819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 479/959] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:41.237896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0e6be-1a3d-42da-bc03-081370eb88f4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.271179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 481/960] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:41 2022] POST /identity/v3/services => generated 242 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:41.272493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.272761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.273045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.292392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:41.298979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:41.364712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.378623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0e6be-1a3d-42da-bc03-081370eb88f4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.390920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 482/961] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:41.418651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.427811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.428028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.428613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.428898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.429168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.438562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.440400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:41.464821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:41.468209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 480/962] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:41.474113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.543719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480e49a-0a68-48bf-81ea-f536ba2f2b0a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.582800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.583614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.583898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.584175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.584836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.586216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:41.601925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:41.603226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:41.604086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:41.604839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 483/963] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:41.608519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82d76-0bc2-4b65-8722-1537fcf0a57e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.613255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.652574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.654083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.654083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.654083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.654471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:41.654799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.656550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.670229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 481/964] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:41.676067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.700636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.704272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.735475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45a5fd-2fa0-4fb0-9841-3e6d85461b56 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.778152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.778845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.779123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.779400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.780103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:41.780406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.781783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.787851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 484/965] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:41.793078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.806538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd4c-cd2b-4cb8-b55f-fc51e5ac85c0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.850451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.851528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.851819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.852098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/46b53ca68e1a4445962f4604d1b90d9c/roles/41fa8686eed24d9f9d00303d8f67e01d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.855204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=46b53ca68e1a4445962f4604d1b90d9c, role_id=41fa8686eed24d9f9d00303d8f67e01d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.856654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934c5a2-9ceb-45cd-b49a-df130f1d3bc6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.910312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f9d4-838c-49f0-89c4-0d7cbee65d69 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:41.945860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:41.947103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:41.947587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:41.948035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:41.949016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:41.949324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:41.950737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:41.952228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:41.956551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 485/966] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:41.963184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:41.995795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 482/967] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:41 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/46b53ca68e1a4445962f4604d1b90d9c/roles/41fa8686eed24d9f9d00303d8f67e01d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:28:42.103763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf589-8f6a-4d7c-85ec-bedbdc8f4233 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:42.153260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:42.153941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:42.154221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:42.154495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:42.155270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:42.159477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272403fb-a94c-4853-b3f4-f839b82a34e3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:42.232757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 486/968] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:28:43.502195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071119-84b3-432a-bf92-f86b181c47cc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:43.502813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071119-84b3-432a-bf92-f86b181c47cc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:43.502813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071119-84b3-432a-bf92-f86b181c47cc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:43.504331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 483/969] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:43.510702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:43.511215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:43.511295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:43.528473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:43.529652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:43.589724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:43.614145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 487/970] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:43.637564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43071119-84b3-432a-bf92-f86b181c47cc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:43.727071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:43.754213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43071119-84b3-432a-bf92-f86b181c47cc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:43.792874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:43.793728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:43.794007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:43.794346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:43.795053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:43.796508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:43.809162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:43.810242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:43.811525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:43.814920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 484/971] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:43.816646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:43.970312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc771e-a3d0-43f9-8eb5-07fe37c3014d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.017587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.018167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.018443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.018741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.019441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:44.019742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.021099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a681438-ee7c-480c-951e-ba33ff32d315 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.027879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 488/972] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:44.035628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.116043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816e621-a637-4143-980b-752d3c886409 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.116464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816e621-a637-4143-980b-752d3c886409 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.116609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816e621-a637-4143-980b-752d3c886409 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.118808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 489/973] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:44.124447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.124698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.124992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.144589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:44.145801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:44.185387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c138b-d631-4507-8471-8266a62fac64 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.199438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.226050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.226747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.227036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.227874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.228610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:44.228929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.230316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.235618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 485/974] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:44.242310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.244972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 490/975] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:44.246863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.265457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.379310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be924c0-3b3b-4944-9db7-f2dc07c4bd5f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.381071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ca7f-f06f-4033-9e38-2c1bc6d2816c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.416483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.416751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.417205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.417311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.417489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.417609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.417765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.417865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.418578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] RBAC: Adding query filter params (interface=public, service_id=86ed0c318a904c6abdd3bf2ad5325155, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:44.418896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.420285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d0c9-9e18-4ef4-978a-6760fc1aeee2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.425537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 486/976] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/endpoints?service_id=86ed0c318a904c6abdd3bf2ad5325155&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:44.426866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd54c-4a32-4a0a-beec-12e59e596dbc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.427155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd54c-4a32-4a0a-beec-12e59e596dbc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.427439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd54c-4a32-4a0a-beec-12e59e596dbc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.428701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 487/977] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:44.431603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:44.431877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bd54c-4a32-4a0a-beec-12e59e596dbc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b1e7bb97-3a10-4c7c-90a6-8fb601b2feae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:44.440879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:44.442134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:44.443040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 491/978] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:44.443858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.443858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.444957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.465655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:44.466914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:44.498232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.507745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.544647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 492/979] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:44.545513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.553881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bd54c-4a32-4a0a-beec-12e59e596dbc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.597069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.597745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.598040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.598308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.599043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.600439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.607870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 488/980] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:28:44.608699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.675431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf227d-ff73-4003-8db1-2f45a4549015 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.726772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.727947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.728242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.728522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.729300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:44.729611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.731140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.746435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 493/981] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:44.746703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad559a8b-46de-4514-b0da-a8f9e3308922 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.747774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.791074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.793180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:44.800363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.801373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.801740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.802140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.803325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.804888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:44.819654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:44.820930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:44.822767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 489/982] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:44.833032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:44.894005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766df8c1-7d67-4038-ab0d-5b80a7bbc350 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.939355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:44.939832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:44.940118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:44.940388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:44.954442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:44.954442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:44.964713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ad2ee-03be-40ee-b8b7-101b199579f6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:44.974343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:44.975494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:44.976398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:44.976398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:44.976398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:44.976398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:44.976398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 494/983] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:44.982096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.021911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.022946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.023503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.023787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.025134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:45.025452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.027903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.034226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 490/984] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:45.040014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.089367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:45.112350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:45.117083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0222602-a7d4-493e-bfb7-40aa85eaae5e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.167021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabb40b-cf23-4b2e-93b7-33bb349f97eb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.172654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.173707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.174161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.174624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.175837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:45.176405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.179184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.208821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 495/985] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:45.209010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.209550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.209834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.210110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.210225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.210855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:45.211161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.217058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.222284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 491/986] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:45.228705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.333966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d01d61-4732-40a3-a2af-cb0516fc1577 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.342170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84da-d568-4837-b22d-69e5a9877709 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.377526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:45.383541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.384624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.396821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.396821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.397055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.397055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.397055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.397055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.397365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.397437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] RBAC: Adding query filter params (interface=admin, service_id=a239b8a804b944a187b2add6075680e8, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:45.397437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.397568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.397648 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.398445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:45.409343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 492/987] 10.222.0.48 () {56 vars in 1249 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/endpoints?service_id=a239b8a804b944a187b2add6075680e8&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:45.410292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:45.417773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:45.426532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 496/988] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:45.428202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.561911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d467-8b5a-4b10-ba94-8f6f48f40c29 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.582657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f6301-5865-4d40-aa52-5ebc23d4ae88 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.602626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.603112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.603478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.603728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.604423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.605812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.622588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 493/989] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/services => generated 1727 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:28:45.647295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.648086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.648429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.648755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.651298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:45.651906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.654456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:45.684436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 497/990] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:45.691606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:45.767532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:45.847225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b50c3-279a-4e77-b389-73d5a5d9b10e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:45.896369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:45.897158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:45.897443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:45.897715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:45.995150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:45.997304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:46.059836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:46.125865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 494/991] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:45 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 10:28:46.131933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:46.222910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:46.280967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4b59d-34d4-4d94-b58b-85b566e9ed7a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:46.320144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:46.320941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.321578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.322348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.339183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:46.341751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:46.361858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:46.362908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:46.363839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:46.363839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:46.363839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:46.363839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:46.363839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 498/992] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:46.371081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:46.501618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf5c9-0738-41f8-ae14-f255cb9f737c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:46.547812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:46.548489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.548848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.549171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.549909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:46.550218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:46.551649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:46.565433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 495/993] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:46.570939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:46.700514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a357aa-3251-4293-a8ed-44a55c3ceed3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:46.711210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:46.744180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:46.745499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.746244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.746618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.753995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:46.755398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-10c349d5-8d53-4e9f-aead-8aa63b1ad0b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:46.770635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:46.771619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:46.772529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:46.774840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 499/994] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:46.779192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:46.796026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:46.826931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88829-a037-46cd-907f-62de05b49287 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.827352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88829-a037-46cd-907f-62de05b49287 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.827533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88829-a037-46cd-907f-62de05b49287 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.829071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 500/995] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:46.835864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.836171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.836432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.854705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:46.855915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:46.909294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0097-b2fb-4e80-96e8-a280faa2cbe7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:46.939513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 501/996] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:46.966834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:46.967512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:46.967788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:46.968067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:46.969178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:46.969415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:46.970163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:46.970900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:46.992565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 496/997] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:46.996221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:47.084486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:47.101842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:47.108099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a886b5-886d-4a5c-8d01-b6589d226942 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:47.122744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a92d5-d893-4c83-b2a3-d3d6189ebdd5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:47.153407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:47.154303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.154620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.154890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.155544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:47.156920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:47.174926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:47.176179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:47.177724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 502/998] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:47.182683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:47.183185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:47.183440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.183680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.183948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.184794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] RBAC: Adding query filter params (role.id=41fa8686eed24d9f9d00303d8f67e01d, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=46b53ca68e1a4445962f4604d1b90d9c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:47.185112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:47.186581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:47.193772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 497/999] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:46 2022] GET /identity/v3/role_assignments?user.id=46b53ca68e1a4445962f4604d1b90d9c&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=41fa8686eed24d9f9d00303d8f67e01d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:47.319014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310638-9ef6-4746-a21d-ad5c6aaf9b14 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:47.379233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:47.379843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.380129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.380411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.381133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:47.381444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:47.383012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:47.389226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 503/1000] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:47.395026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:47.518180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cacb5a-22b4-4349-b9a2-183375fd2d78 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:47.566804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:47.567879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.568337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.568781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.569918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:47.570365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:47.573274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d9142-3422-4de8-a1f5-cc7cf01b5740 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:47.576799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:47.581896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 498/1001] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 21 10:28:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:47.590117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:47.631103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:47.713949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef66d7-73d7-45a5-a0a3-baf0797c9a12 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:47.752574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:47.753247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.753536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.753824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.754569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:47.755923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:47.804820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357b5be-e8bb-4e0e-b380-47969f6cd54a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.805132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357b5be-e8bb-4e0e-b380-47969f6cd54a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.805985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357b5be-e8bb-4e0e-b380-47969f6cd54a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.807530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 499/1002] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:47.813057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:47.813342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:47.813624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:47.832133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:47.838037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:47.866579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 504/1003] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:28:47.931105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 500/1004] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:47 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:47.954754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:48.056423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:48.078153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8ae2f-d268-4dc9-bf87-d98da6b20af2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:48.118006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:48.118974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:48.119256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:48.121035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:48.121035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:48.121916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:48.135152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:48.136227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:48.137076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 505/1005] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 21 10:28:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:48.145116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:48.270256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67418b3-5f7e-492f-8382-63f47e712b54 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:48.308028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:48.319213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:48.319901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:48.320184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:48.320459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:48.321527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:48.321858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:48.323744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:48.337491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 501/1006] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:48.339582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:48.484114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21980c28-c4e4-4d12-bfcd-df8f685d2d8d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:48.522827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:48.523420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:48.523703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:48.523977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:48.524813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:48.525092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:48.526435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937b4f-0977-4a73-9196-bf36698bc7bc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:48.532096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 506/1007] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 21 10:28:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:48.539301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:48.662037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bad5b-483a-4c9f-b6b6-823d570473d9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:48.701230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:48.702213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:48.702747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:48.703227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:48.704176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:48.705749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47984b9-60ac-401f-ab4e-2ae45f708c92 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:48.780847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 502/1008] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:28:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:49.416431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0e0b-fb57-4ac8-a390-8c3c6bf2e537 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:49.416959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0e0b-fb57-4ac8-a390-8c3c6bf2e537 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:49.416959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0e0b-fb57-4ac8-a390-8c3c6bf2e537 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:49.418265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 507/1009] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:49.424394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:49.424702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:49.424955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:49.443797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:49.445002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:49.503786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:49.555150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 503/1010] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:49 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:49.578898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0e0b-fb57-4ac8-a390-8c3c6bf2e537 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:49.594914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:49.715724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0e0b-fb57-4ac8-a390-8c3c6bf2e537 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:49.763347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:49.764165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:49.764446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:49.764838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:49.765503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:49.766936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:49.782537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:49.783678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:49.784526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:49.786292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 508/1011] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 21 10:28:49 2022] GET /identity/v3/services/image => generated 86 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:49.792277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:49.922806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae6881-a281-410a-b444-7c6a8ce6d27c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:49.963964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:49.964975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:49.965435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:49.965937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:49.966867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] RBAC: Adding query filter params (name=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:49.967349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:49.968925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007c00-3c2a-4cf2-be16-3b273c495f89 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:49.986015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 504/1012] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:49.988089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:50.112284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:50.141408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd025-cd4f-4447-97eb-ac83980223a7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:50.157372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ea8c-2c02-4296-aebb-0801b8f1602d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.157800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ea8c-2c02-4296-aebb-0801b8f1602d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.157917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ea8c-2c02-4296-aebb-0801b8f1602d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.159361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 505/1013] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:50.165740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.166026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.166357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.189970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:50.189970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.189970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.190329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.190989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:50.191297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:50.192668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:50.198122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:50.199670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:50.210833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 509/1014] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:50.255097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:50.300173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 506/1015] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:50.324954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:50.454231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c72a9-8fa9-4f19-a19a-6dbff4a81a05 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:50.501246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:50.502447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.502980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.503457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.512560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:50.517595 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:50.526202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:50.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:50.528181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 510/1016] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:50.536420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:50.651070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603adb6a-f737-4bc9-8802-44feb4b0ac69 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:50.691523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:50.693114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.693114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.693114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.693500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:50.693801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:50.695223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:50.705217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 507/1017] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:28:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:50.711603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:50.849844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a5c7-6a76-4af7-8cfc-181be08b8db3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:50.888824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:50.889642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:50.889927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:50.890200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:50.901487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:50.902974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3185259f-5701-47ab-93f4-975c74f46606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:50.926773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:50.927812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:50.928790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:50.928790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:50.928790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:50.928790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:50.928790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 511/1018] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:50.935505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:50.971079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.049514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e86db2-1dad-42b6-bb1d-770101633c66 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.049867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e86db2-1dad-42b6-bb1d-770101633c66 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.050067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e86db2-1dad-42b6-bb1d-770101633c66 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.051471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 512/1019] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:51.057793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.058096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.058342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.062944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db235695-7d69-44e3-a7cc-73538bfb241f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.082116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:51.083457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:51.105297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.105882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.106196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.106483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.107237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:51.107544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.108919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.128421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 508/1020] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:51.129511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:51.143967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.172425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 513/1021] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:51 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:51.196915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:51.250987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.261544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dddc9-4e28-4256-9992-a1a327d7166b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.307091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.307897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.308180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.308455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.315523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ef7f9-7b2c-457c-80f9-70355e6c7943 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.316981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.318451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:51.335361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:51.337287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:51.339270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 509/1022] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:28:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:51.343169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:51.361480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.362508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.362981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.363252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.364436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.365866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.367542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 514/1023] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:51.374196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:51.452926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.465311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d7def-b42c-4326-92d5-da97d65c0e19 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.505259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.506291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.506602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.506879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.507740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:51.508056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.510004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.513476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733868af-bdf0-4f95-ad1e-9df49841e272 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.532138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 510/1024] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:28:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:51.538514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:51.558736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.561582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.562266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.562569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.563174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.563569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.565636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.655505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 515/1025] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:51 2022] POST /identity/v3/projects => generated 311 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:28:51.671575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c8da-712c-44c3-9e7f-3f3335ce78c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:51.715061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:51.715770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:51.716060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:51.716339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:51.809548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:51.811116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:51.824763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:51.946649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 511/1026] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:51 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:51.952289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:52.064660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:52.069760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746f81d-271f-4c40-97f2-9e59e2f0978c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:52.116521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:52.117330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:52.117616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:52.118949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:52.130293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:52.131810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:52.163060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:52.164054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:52.165698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:52.165698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:52.165698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:52.165698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:52.165698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 516/1027] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:52.174083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:52.297136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e6e27-9c4c-4979-ba75-23ff025d8d7a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:52.309662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:52.338179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:52.338908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:52.339190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:52.339463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:52.340206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:52.340513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:52.341909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:52.362878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 512/1028] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:52.365570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:52.487231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d023d8-3237-43d4-abc0-743dee349353 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:52.531536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:52.532350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:52.532629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:52.532922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:52.542456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:52.543942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:52.559735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:52.560671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:52.566651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 517/1029] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:52.568703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:52.712543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbbe0-12f1-4530-9acc-84f44d3a77da None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:52.755070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:52.755773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:52.756056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:52.756335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:52.757144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:52.757460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:52.759225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6f75-73a6-4860-a64f-0d552606ae7f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:52.772897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 513/1030] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:52.780038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:52.801964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:52.917479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1054dcd-2cbc-4725-9dff-95fd5d9fd494 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:52.969003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:52.970109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:52.970615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:52.970937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:52.971790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] RBAC: Adding query filter params (role.id=41fa8686eed24d9f9d00303d8f67e01d, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:52.972104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:52.973560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a2c77-6bf8-402d-b4f6-8eea8436962a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:52.981236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 518/1031] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:52 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=41fa8686eed24d9f9d00303d8f67e01d => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:53.127698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845583d3-b88e-4dbf-a533-7e9d8ad361f8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:53.128265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845583d3-b88e-4dbf-a533-7e9d8ad361f8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:53.128343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845583d3-b88e-4dbf-a533-7e9d8ad361f8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:53.129881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 514/1032] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:53.136097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9b9f-41fd-4e6c-bb4a-4e864d22de4f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:53.136416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9b9f-41fd-4e6c-bb4a-4e864d22de4f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:53.136656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9b9f-41fd-4e6c-bb4a-4e864d22de4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:53.155574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f20b9b9f-41fd-4e6c-bb4a-4e864d22de4f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:53.156816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f20b9b9f-41fd-4e6c-bb4a-4e864d22de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:53.246670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 519/1033] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:53.264892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845583d3-b88e-4dbf-a533-7e9d8ad361f8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:53.298399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:53.385225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845583d3-b88e-4dbf-a533-7e9d8ad361f8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:53.428385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:53.429050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:53.429329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:53.429602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:53.430927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:53.432544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314e048-3f85-498c-9cc8-fcfca0aef865 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:53.533889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 515/1034] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:28:53 2022] POST /identity/v3/services => generated 242 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:54.502683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935ba5-954f-444e-ba2d-4b0a095fa3ae None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:54.503209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935ba5-954f-444e-ba2d-4b0a095fa3ae None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:54.503209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935ba5-954f-444e-ba2d-4b0a095fa3ae None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:54.504677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 520/1035] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:54.510595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:54.511065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:54.511149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:54.528674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:54.529872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:54.642609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 516/1036] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:54.667941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42935ba5-954f-444e-ba2d-4b0a095fa3ae None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:54.679158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:54.803857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42935ba5-954f-444e-ba2d-4b0a095fa3ae None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:54.844868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:54.845722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:54.846006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:54.846278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:54.847511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:54.850104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3ae72-2342-47ab-a6bb-1bfca3f23338 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:54.858739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 521/1037] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:54.862422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:54.881256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:54.969138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c56f8-2edc-4df9-ae73-af8293bb91e1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:55.005835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:55.006603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.006882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.007177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.007862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:55.009366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a09b5-a136-4f8a-8e3e-f5841d94d0a2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:55.110666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 517/1038] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:28:54 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:28:55.129914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e4e3-1a29-4103-9359-d27aa0b7fc16 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.134301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e4e3-1a29-4103-9359-d27aa0b7fc16 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.134301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e4e3-1a29-4103-9359-d27aa0b7fc16 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.134301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 522/1039] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:55.139844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.140272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.140845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.165750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:55.167233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:55.255308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 518/1040] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:55.278902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61e4e3-1a29-4103-9359-d27aa0b7fc16 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:55.374606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:55.409177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61e4e3-1a29-4103-9359-d27aa0b7fc16 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:55.464883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:55.465718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.466001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.466278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.482657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:55.492813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:55.495368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:28:55.497109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:55.498612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 523/1041] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:55.501910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:55.504443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:55.618251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b737e-ba38-4aba-a4eb-e3c993c05ef5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:55.657749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:55.658441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.658757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.659035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.659793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:55.660104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:55.661524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd512b-cbc0-49ae-947f-a77a3d1b59b6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:55.677994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 519/1042] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:28:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:55.680085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:55.690588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2d705-f501-4a04-85a8-d1d932670f36 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.690922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2d705-f501-4a04-85a8-d1d932670f36 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.691146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2d705-f501-4a04-85a8-d1d932670f36 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.692823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 520/1043] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:28:55.699149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.699540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.699830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.721877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:55.723125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:55.793939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:55.808610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001911-547d-4957-ba76-c76d75976ba2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:55.810437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 521/1044] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:55.828272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:55.847705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:55.848913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.849279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:55.849566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:55.851878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:55.864249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:55.868271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:55.891860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:55.892872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:55.893842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:55.893842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:55.893842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:55.893842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:55.894707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 524/1045] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:55.899569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:55.950194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299e4d8-825c-42b5-b986-22ee6e6947f7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:55.997988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:55.999344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:55.999821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.000281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.001092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.002490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:56.016844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:56.017987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:56.018944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 522/1046] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 21 10:28:55 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:56.025149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.032673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffcf5a-2fd5-4d22-b0e7-d45ec5fa59b9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.083180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.083853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.084139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.084417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.085136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.085443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.087615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.098781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.100306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 525/1047] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:56.107465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.162734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9273c-a24b-4692-a596-3b3ada3980b6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.169780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.207341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.208595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.209083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.209545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.210456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.210796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.213270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.224807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 523/1048] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:56.225574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.237468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23049b1-c560-4784-8f68-3333fa1635ee None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.277310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.278112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.278396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.278723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.286220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.287708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:28:56.320607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:56.322356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:56.328781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 526/1049] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:56.329422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.361586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3345049-8589-476c-b8d0-b085e105bbb4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.403939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.419774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.422715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.422715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.422715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.422715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] RBAC: Adding query filter params (type=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.422715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.423559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.428848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 524/1050] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:56.434871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.455530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226413d-df18-4adf-9db7-1c91bffaac71 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.465491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.500067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.500744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.501026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.501301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.502584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.502905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.504333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.523794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 527/1051] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:56.529701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.566464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aec3e5-8ecd-474f-920c-69a030581178 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.605007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.605670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.605962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.606224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.607078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] RBAC: Adding query filter params (interface=public, service_id=1ef9de2cacd744be8d9cae7c88600ded, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.607387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.608769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.619364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 525/1052] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/endpoints?service_id=1ef9de2cacd744be8d9cae7c88600ded&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:56.620447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.651791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abdacd-2235-4bb9-94c8-2b559cc11dd1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.674693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.692941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.693758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.694027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.694302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.705397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.706884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:28:56.725555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:56.726824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:56.727923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:28:56.727923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:28:56.727923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:28:56.727923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:56.727923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 528/1053] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:56.732666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:56.734920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b3d12-0205-48c3-a430-87af4195da5e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.753218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:56.776253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.776975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.777250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.777496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.778187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.779619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.788155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 526/1054] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:28:56.882228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a74b2-59b8-4349-baf9-48d205841876 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:56.923266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:56.923939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:56.924219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:56.924492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:56.925344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:56.925654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:56.927090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:56.945632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 529/1055] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:56.947837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.064306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425b8a-b07d-4c1b-b6b5-8671e6260701 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:57.079391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:57.101675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:57.103242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.103521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.103790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.112761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:57.114221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:28:57.129979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:57.131699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:28:57.132666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:28:57.134614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 527/1056] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:57.140413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.244319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:57.268201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426eb7-7523-4d1d-90a8-628acbf188f8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:57.307647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:57.308314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.308593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.308865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.309645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:57.309948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:57.311391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fa878-a758-4f09-8dd0-b74ab9a81d49 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:57.327035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 530/1057] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:57.332790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.366724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a2f-b17a-4abe-b915-48268b4e966c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.366724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a2f-b17a-4abe-b915-48268b4e966c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.366724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a2f-b17a-4abe-b915-48268b4e966c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.366724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 531/1058] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:57.372727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.373198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.373478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.390888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:57.392167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:57.460106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5057d-5719-47ff-bad6-021a3f2289d9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:57.470445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 532/1059] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:57.496518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.504621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:57.505513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.505789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.506057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/41fa8686eed24d9f9d00303d8f67e01d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.509842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=23fabbe0e78d45f890b21fcf4b348721, role_id=41fa8686eed24d9f9d00303d8f67e01d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:57.512051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:57.571579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:57.595791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 528/1060] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:28:57 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/23fabbe0e78d45f890b21fcf4b348721/roles/41fa8686eed24d9f9d00303d8f67e01d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:28:57.611385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9a9b8-767c-46f9-9e53-5000be926983 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:57.654153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:57.655013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.655301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.655570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.668904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:28:57.676798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:57.678419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:57.679539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 533/1061] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:28:57.686680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.731191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:57.810965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9056-329c-43d2-9b78-2ec23ef3dd4f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:57.848400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:57.849057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:57.849346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:57.849626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:57.853999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:57.853999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:57.853999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:57.862596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 529/1062] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:57.868832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:57.977619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a22ee0-b9e9-49df-9490-9d2a1e974a2d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:58.007184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:58.020738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:58.025119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:58.025119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:58.025119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] PATH_INFO: `/v3/users/e0e3a72c4d054c83bb9e92241ad67686` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:58.039121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] RBAC: Authorizing `identity:get_user(user_id=e0e3a72c4d054c83bb9e92241ad67686)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:58.041882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:58.046631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 534/1063] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:28:57 2022] GET /identity/v3/users/e0e3a72c4d054c83bb9e92241ad67686 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:58.050877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:58.156830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:58.168273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a422f-4e70-4d16-ac4d-f638b3f46477 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:58.209974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:58.210654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:58.210934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:58.211204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:58.293694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:58.295192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:58.397825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:58.405601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 530/1064] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:28:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:28:58.411892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:58.529077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f3a7c-db9b-4a38-aba4-44e3e96d6060 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:58.567858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:58.568724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:58.569000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:58.569274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:58.570634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:58.572254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fe413-9713-4059-981b-50fd99d4b873 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:58.578999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 535/1065] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:28:58 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:58.580261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:58.699983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dda9a-b65f-43eb-a462-e662615008dc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:58.745231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:58.746182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:58.746457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:58.746766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:58.747640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=d6d05c688e774a07b14b8e45c2d9d688, user.id=e0e3a72c4d054c83bb9e92241ad67686) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:58.747953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:58.749463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d272f-2507-400d-8ce5-08d224b22270 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:58.762646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 531/1066] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:28:58 2022] GET /identity/v3/role_assignments?user.id=e0e3a72c4d054c83bb9e92241ad67686&scope.project.id=d6d05c688e774a07b14b8e45c2d9d688&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:28:59.392481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1aa1-85ba-486b-bebe-8e0211a30e3f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.393009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1aa1-85ba-486b-bebe-8e0211a30e3f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.393009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1aa1-85ba-486b-bebe-8e0211a30e3f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.394694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 536/1067] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:59.400401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83234-89fa-4cf6-b73a-90ea359a8e86 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.400687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83234-89fa-4cf6-b73a-90ea359a8e86 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.400961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83234-89fa-4cf6-b73a-90ea359a8e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.418804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e3d83234-89fa-4cf6-b73a-90ea359a8e86 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:59.419936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e3d83234-89fa-4cf6-b73a-90ea359a8e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:59.489967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:59.503916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 532/1068] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:28:59.530119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d1aa1-85ba-486b-bebe-8e0211a30e3f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:59.599471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:59.662907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d1aa1-85ba-486b-bebe-8e0211a30e3f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:59.703282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:59.703491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d8aa9-4e61-4208-9420-248d6b2beced None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.704025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d8aa9-4e61-4208-9420-248d6b2beced None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.704025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d8aa9-4e61-4208-9420-248d6b2beced None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.704159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.704318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.704699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] PATH_INFO: `/v3/services/image` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.705185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:59.705904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 533/1069] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:28:59.706565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:59.713219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.713485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.713767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:28:59.723708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:28:59.724780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 10:28:59.725640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:28:59.727146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 537/1070] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 21 10:28:59 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:28:59.731398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:28:59.731600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:59.732594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:28:59.822119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 534/1071] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:28:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:28:59.848898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:59.868319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63e742-d4aa-4f2d-9f4a-021341918801 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:59.887711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:28:59.910395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:28:59.911409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:28:59.911691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:28:59.911966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:28:59.912892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] RBAC: Adding query filter params (name=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:28:59.913641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:28:59.915671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:28:59.926617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 538/1072] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:28:59.927379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:28:59.969052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dd5c8-eeb1-4d3d-87cc-3aea35f6afaf None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:28:59.994600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:00.010745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.011564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.011845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.012118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.023310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:00.032000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:00.033069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 10:29:00.033904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:00.037236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 535/1073] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 21 10:28:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:00.043910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.060606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2755c-f955-42ea-bb3e-bbf1028ee71e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.099225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.099894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.100178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.100454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.101161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:00.101469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.103344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.109017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 539/1074] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:28:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:00.115499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.179192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:00.197417 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a5e19-24e6-4f5c-8c20-67f9a35a7fde None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.240351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.241034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.241336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.241693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.242441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:00.243305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.245918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.251038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe0bf9-33a9-4e11-b5f7-20c4a0b69867 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.259927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 536/1075] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:29:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:00.262670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.297149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.298270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.299020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.299709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.300774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.302916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.315400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:00.377058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c07-4fdc-40ab-89c2-6bb7ad070d28 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.385884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 540/1076] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:29:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:29:00.427147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.427955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.428239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.428515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.442744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.444235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:29:00.468420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:00.469423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:00.470324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:29:00.470324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:29:00.470324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:00.470324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:00.470324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 537/1077] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:00.475913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.601189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:00.606103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da542187-8f1b-44b4-9944-4d9d81a9ce97 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.648297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.649851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.650156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.650445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.651246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:00.651558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.652958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.665905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 541/1078] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:00.671688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.777086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:00.788102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf16ac-5674-4067-b4e0-318ba2517de8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:00.832903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:00.833706 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.833985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.834259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.843251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:00.844713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-56912e32-59f0-43b8-bfa9-c1527928a9b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:29:00.861432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:00.862428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:00.863422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 538/1079] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 21 10:29:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:00.871453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:00.887143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88d2b-2d90-4d9d-b144-4581d902967a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.887548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88d2b-2d90-4d9d-b144-4581d902967a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.887709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88d2b-2d90-4d9d-b144-4581d902967a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.889294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 539/1080] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:29:00.894478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:00.894792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:00.895073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:00.912712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:00.913906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:00.996625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312cbe-afa8-4ae6-9b40-6e9b4e2576b0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.005008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 540/1081] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:01.028070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.041845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.042532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.042876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.043162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.044006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:01.044317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.045818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.065913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 542/1082] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 21 10:29:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:01.072347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.096405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.135091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.159689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789239bd-beca-447f-94c4-596850b28cf6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.204453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b937-ac58-4618-b665-5a1c80a04643 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.216904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.217733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.218017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.218289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.231956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:01.239850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:01.240841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:01.241700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 541/1083] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:01.247771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.254437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.255310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.255762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.256204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.340903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.342499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.364258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.371438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770f0df-f2e9-456e-862a-e22ab3985536 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.412670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.413352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.413637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.413915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.415351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:01.415675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.417227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.428388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 542/1084] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:01.434606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.437693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.473825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 543/1085] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 10:29:01.479529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.574768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2131-7cc9-43fe-b8c1-f3e12687ddc4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.613587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d49c37-684b-47a3-a146-1a6702c8d17e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.616650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.617567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.617849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.618124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] PATH_INFO: `/v3/users/e0e3a72c4d054c83bb9e92241ad67686` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.619391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] RBAC: Authorizing `identity:get_user(user_id=e0e3a72c4d054c83bb9e92241ad67686)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.621102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.622976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 543/1086] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/users/e0e3a72c4d054c83bb9e92241ad67686 => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:01.628929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.642181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.667174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.668689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.669203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.669678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.687017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.688794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.706057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:29:01.713727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:01.721082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:01.722834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:29:01.722834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:29:01.722834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:01.722834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:01.722834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 544/1087] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:01.725130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.741540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dec54-7db7-44fc-be57-d5729a74d02b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.785850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.786741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.787019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.787277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.788463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.790768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.797804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 544/1088] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:01.798608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.853251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93dc3a-739f-4458-8563-bdb18aac11e7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.895031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.895721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.896004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.896296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.897007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:01.897311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.898769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:01.911300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 545/1089] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:01.917537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:01.928220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b3d-8861-4053-bc97-831cb9041275 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:01.951646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:01.967081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:01.967975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:01.968256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:01.968572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688/users/e0e3a72c4d054c83bb9e92241ad67686/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:01.970574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6d05c688e774a07b14b8e45c2d9d688, user_id=e0e3a72c4d054c83bb9e92241ad67686, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:01.974743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:02.025289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:02.031343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 545/1090] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:29:01 2022] PUT /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688/users/e0e3a72c4d054c83bb9e92241ad67686/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:29:02.046370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd86c2-d686-42e4-a1be-3dce8e1b0366 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:02.091564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:02.092480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.092842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.093323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:02.103413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:02.104788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:29:02.120330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:02.122254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:29:02.124643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:02.126408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 546/1091] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:02.128217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:02.264908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499146b-353a-4b19-90dd-4a1e2a8c6981 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:02.314282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:02.314995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.315281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.315558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:02.316354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:02.316667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:02.318126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe3532-928d-4c9c-ad45-a59ba2286bd4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:02.338602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 546/1092] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:02.341285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:02.443277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:02.454091 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc051d6b-eb40-4eb5-9cba-6671f598037d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:02.491755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:02.492402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.493374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.493656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:02.494490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] RBAC: Adding query filter params (role.id=41fa8686eed24d9f9d00303d8f67e01d, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=23fabbe0e78d45f890b21fcf4b348721) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:02.494830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:02.496530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99eed1-951d-4298-93bc-cee592e3814a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:02.506440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 547/1093] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:02 2022] GET /identity/v3/role_assignments?user.id=23fabbe0e78d45f890b21fcf4b348721&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=41fa8686eed24d9f9d00303d8f67e01d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:02.588061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f891a09-f5c4-46e0-b9db-70c6f0d24c02 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.588552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f891a09-f5c4-46e0-b9db-70c6f0d24c02 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.589083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f891a09-f5c4-46e0-b9db-70c6f0d24c02 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:02.592972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 547/1094] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:29:02.598044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.598394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.598693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:02.619240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:02.620766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:02.706600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 548/1095] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:02.733024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f891a09-f5c4-46e0-b9db-70c6f0d24c02 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:02.811934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:02.871293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f891a09-f5c4-46e0-b9db-70c6f0d24c02 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:02.916855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:02.917532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:02.917817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:02.918094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:03.009057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:03.010531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:03.072937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:03.121300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 548/1096] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:03.127364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:03.223973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:03.246521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc9890-f8f0-42c4-8089-bf4df433e3d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:03.285633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:03.286349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:03.286659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:03.286935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:03.297928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:03.299414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-afe0003d-05a3-45be-9c77-2d88f685ed26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:29:03.316819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:03.317748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:03.318673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:29:03.318673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:29:03.318673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:29:03.318673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:03.318673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 549/1097] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:03.324241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:03.432991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c7f58-5c2b-4fa1-8e79-d2d284677981 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:03.471976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:03.472662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:03.472938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:03.473206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:03.473922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:03.474223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:03.475608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde6059-f625-4bc3-80e4-de817e4eb085 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:03.498576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 549/1098] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:04.161755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b2df-6e5e-4ad3-93e8-e4e4dd6d1943 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.162290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b2df-6e5e-4ad3-93e8-e4e4dd6d1943 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.162290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b2df-6e5e-4ad3-93e8-e4e4dd6d1943 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:04.164035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 550/1099] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:04.170138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.170619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.170773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:04.188411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:04.189638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:04.205148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:04.263256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 550/1100] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:04.287580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00b2df-6e5e-4ad3-93e8-e4e4dd6d1943 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:04.400263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00b2df-6e5e-4ad3-93e8-e4e4dd6d1943 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:04.436393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:04.437104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.437383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.437657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:04.440537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:04.449108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:04.456800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:04.457785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:04.458820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 551/1101] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:04.464694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:04.607178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a5e0c-83be-415a-a364-affddf8f6cf3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:04.645130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:04.645809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.646089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.646360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:04.647145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:04.647454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:04.648836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:04.657670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 551/1102] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:04.663767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:04.772199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6eb35-579f-4f05-913a-ad129d33108c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:04.807467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:04.808345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.808684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.808894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] PATH_INFO: `/v3/users/e0e3a72c4d054c83bb9e92241ad67686` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:04.810085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] RBAC: Authorizing `identity:get_user(user_id=e0e3a72c4d054c83bb9e92241ad67686)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:04.811540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:04.813175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 552/1103] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:04 2022] GET /identity/v3/users/e0e3a72c4d054c83bb9e92241ad67686 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:04.819202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:04.832797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:04.935270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883ad1e-4393-40bf-bc7f-98b3ecc86058 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:04.976123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:04.976123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:04.976123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:04.976123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:05.048747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:05.050124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:05.062914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:05.160062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 552/1104] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:05.166989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:05.231782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:05.279177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031b3c3-f557-442e-a9b9-eb627dc54e9d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:05.315730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:05.316520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:05.316807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:05.317084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:05.318316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:05.319807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78666170-da0b-4ae6-9811-31a169a17974 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:05.322165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 553/1105] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:05 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:05.328696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:05.440759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f76b8a-22c5-4a7d-a381-56e11cd57411 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:05.455255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:05.481087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:05.481769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:05.482062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:05.482342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:05.483235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=d6d05c688e774a07b14b8e45c2d9d688, user.id=e0e3a72c4d054c83bb9e92241ad67686) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:05.483555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:05.485032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e452-aa8f-4b6c-a1f8-1b23ed4c3259 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:05.491958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 553/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:05 2022] GET /identity/v3/role_assignments?user.id=e0e3a72c4d054c83bb9e92241ad67686&scope.project.id=d6d05c688e774a07b14b8e45c2d9d688&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:07.615059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2df0-7dc0-42ed-9dd4-4980e1d885a0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:07.615599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2df0-7dc0-42ed-9dd4-4980e1d885a0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:07.615599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2df0-7dc0-42ed-9dd4-4980e1d885a0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:07.616906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 554/1107] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:07.623709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:07.624237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:07.624237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:07.642847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:07.644243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:07.720302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 554/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:07.746953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd2df0-7dc0-42ed-9dd4-4980e1d885a0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:07.857186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd2df0-7dc0-42ed-9dd4-4980e1d885a0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:07.888883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:07.894184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:07.895021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:07.895306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:07.895584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:07.896797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:07.898243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599186e-807d-4ebb-a9ad-d55e8ffe504b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:07.900576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 555/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:07.906578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:08.016475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b62baa-3a7c-4533-9a17-4320d50fb56a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:08.019003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:08.054091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:08.054813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:08.055284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:08.055284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:08.056376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:08.057821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081522e-ec57-401c-8fc3-e845e28cca94 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:08.151160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 555/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:29:07 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:29:10.316070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775440f-033a-4111-9eeb-4926c2f3a0ec None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:10.316624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775440f-033a-4111-9eeb-4926c2f3a0ec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:10.316624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775440f-033a-4111-9eeb-4926c2f3a0ec None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:10.317974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 556/1111] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:10.323914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:10.324372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:10.324462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:10.341548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:10.342782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:10.412590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 556/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:10.438135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775440f-033a-4111-9eeb-4926c2f3a0ec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:10.546054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775440f-033a-4111-9eeb-4926c2f3a0ec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:10.582498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:10.583379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:10.583662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:10.583938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] PATH_INFO: `/v3/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:10.585403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] RBAC: Authorizing `identity:get_role(role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:10.587204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:10.588584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 557/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:10 2022] GET /identity/v3/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:10.594483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:10.686500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:10.704261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149254-61f0-4c80-8e18-b425a1d9c670 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:10.730339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:10.731313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:10.731588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:10.731859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] PATH_INFO: `/v3/users/86373c750abf42ab9750925711ecc179` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:10.747232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] RBAC: Authorizing `identity:get_user(user_id=86373c750abf42ab9750925711ecc179)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:10.752561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:10.752561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 557/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:10 2022] GET /identity/v3/users/86373c750abf42ab9750925711ecc179 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:10.756305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:10.824241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:10.857898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91e533-9781-46ca-a592-1714ed9e52a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:10.894389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:10.895117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:10.895424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:10.895623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:10.972869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:10.974234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:11.040705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:11.074675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 558/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:11.080688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:11.186409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939952-ed0f-40e8-9a44-80f9713f81a6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:11.221991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:11.222941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:11.223171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:11.223447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:11.224645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:11.226090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d182b1c-c677-438e-9774-2f3e616b43f6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:11.228389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 558/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:11 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:11.234629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:11.337936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a7a0e-0404-475b-8278-5e2dd6e12fe5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:11.373877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:11.374524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:11.374831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:11.375109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:11.375925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=d6d05c688e774a07b14b8e45c2d9d688, user.id=86373c750abf42ab9750925711ecc179) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:11.376236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:11.377647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfec29-b70a-47d5-897f-682ece1ab94b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:11.383560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 559/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:11 2022] GET /identity/v3/role_assignments?user.id=86373c750abf42ab9750925711ecc179&scope.project.id=d6d05c688e774a07b14b8e45c2d9d688&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:13.427458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32bb1-3f7f-4cf6-a606-0a9ef4fb0c3c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:13.427989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32bb1-3f7f-4cf6-a606-0a9ef4fb0c3c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:13.427989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32bb1-3f7f-4cf6-a606-0a9ef4fb0c3c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:13.429307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 559/1118] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:13.435649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:13.436093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:13.436208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:13.453211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:13.454422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:13.481729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:13.525764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 560/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:13.550014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f32bb1-3f7f-4cf6-a606-0a9ef4fb0c3c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:13.569106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:13.654808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f32bb1-3f7f-4cf6-a606-0a9ef4fb0c3c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:13.690229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:13.691048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:13.691328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:13.691602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] PATH_INFO: `/v3/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:13.693114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] RBAC: Authorizing `identity:get_role(role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:13.694480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:13.695983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 560/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:13 2022] GET /identity/v3/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:13.701868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:13.807655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e5e86-638a-40fe-bbb9-f193c2657eb5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:13.845166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:13.846037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:13.846322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:13.846628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] PATH_INFO: `/v3/users/86373c750abf42ab9750925711ecc179` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:13.847821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] RBAC: Authorizing `identity:get_user(user_id=86373c750abf42ab9750925711ecc179)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:13.849261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:13.850925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 561/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:13 2022] GET /identity/v3/users/86373c750abf42ab9750925711ecc179 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:13.857705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:13.941030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:13.964297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ba7f-60f7-4429-9765-1f1b0d89f556 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:14.000990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:14.001857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:14.002140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:14.002416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:14.003632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:14.005077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f268cd-20a5-4be9-99c5-add14296b7f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:14.006833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 561/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:13 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:14.013277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:14.044504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:14.114408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ffe98-8308-44b8-9825-bd4703770df2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:14.150625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:14.151517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:14.151791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:14.152061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688/users/86373c750abf42ab9750925711ecc179/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:14.154231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] RBAC: Authorizing `identity:create_grant(project_id=d6d05c688e774a07b14b8e45c2d9d688, user_id=86373c750abf42ab9750925711ecc179, role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:14.155678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba20527-9a5d-47e7-ba2e-9fc4f025886e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:14.213463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 562/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:29:14 2022] PUT /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688/users/86373c750abf42ab9750925711ecc179/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:29:16.317889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddaf4e-ae87-4457-ad5a-f76db44129e9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:16.318434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddaf4e-ae87-4457-ad5a-f76db44129e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:16.318434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddaf4e-ae87-4457-ad5a-f76db44129e9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:16.319977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 562/1124] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:16.326339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:16.326827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:16.326921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:16.343986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:16.345165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:16.415624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 563/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:16.440687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddaf4e-ae87-4457-ad5a-f76db44129e9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:16.544764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddaf4e-ae87-4457-ad5a-f76db44129e9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:16.582581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:16.583431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:16.583714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:16.583990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] PATH_INFO: `/v3/roles/ed2f59b341da49c4be7e60232c2a8411` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:16.585454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] RBAC: Authorizing `identity:get_role(role_id=ed2f59b341da49c4be7e60232c2a8411)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:16.587080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:16.588479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 563/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:16 2022] GET /identity/v3/roles/ed2f59b341da49c4be7e60232c2a8411 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:16.594286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:16.616077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:16.704951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23cb2-51ce-4f8a-b6f8-689aaf42e1bc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:16.742212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:16.743104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:16.743388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:16.743664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] PATH_INFO: `/v3/users/86373c750abf42ab9750925711ecc179` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:16.744867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] RBAC: Authorizing `identity:get_user(user_id=86373c750abf42ab9750925711ecc179)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:16.746568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:16.748236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 564/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:16 2022] GET /identity/v3/users/86373c750abf42ab9750925711ecc179 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:16.754228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:16.777000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:16.862315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113f57b-44ec-4e29-8eb4-d351aa859d27 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:16.896666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:16.897314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:16.897597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:16.897872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:16.969266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:16.970650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:16.992191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:17.077851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 564/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:17.083846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:17.174258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:17.186384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4054ae-2864-4019-b501-3aa877278a46 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:17.223959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:17.224732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:17.225016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:17.225287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] PATH_INFO: `/v3/projects/d6d05c688e774a07b14b8e45c2d9d688` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:17.226506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d05c688e774a07b14b8e45c2d9d688)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:17.228158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc75ee-3259-418f-aca1-8ed5b04957f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:17.229860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 565/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:17 2022] GET /identity/v3/projects/d6d05c688e774a07b14b8e45c2d9d688 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:17.236100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:17.335667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e85a-98c5-4531-8a6a-d8b6a5f88025 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:17.358031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:17.372474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:17.373139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:17.373423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:17.373702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:17.374559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] RBAC: Adding query filter params (role.id=ed2f59b341da49c4be7e60232c2a8411, scope.project.id=d6d05c688e774a07b14b8e45c2d9d688, user.id=86373c750abf42ab9750925711ecc179) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:17.374882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:17.376327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064acaf-1cdc-412b-a111-b88c6cc1f8b9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:17.384272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 565/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:17 2022] GET /identity/v3/role_assignments?user.id=86373c750abf42ab9750925711ecc179&scope.project.id=d6d05c688e774a07b14b8e45c2d9d688&role.id=ed2f59b341da49c4be7e60232c2a8411 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:19.575878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547bbec-778e-4116-be15-1ab8f96fa707 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:19.576425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547bbec-778e-4116-be15-1ab8f96fa707 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:19.576425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547bbec-778e-4116-be15-1ab8f96fa707 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:19.578133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 566/1131] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:29:19.584383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:19.584927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:19.585014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:19.606569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:19.607834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:19.682798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 566/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:19.707281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547bbec-778e-4116-be15-1ab8f96fa707 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:19.813741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547bbec-778e-4116-be15-1ab8f96fa707 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:19.857394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:19.858117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:19.858401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:19.858715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:19.859905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:19.861342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6e8ed-79c7-416e-8c0c-53d2280e79da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:19.863010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 567/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:19.869658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:19.973983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e8957-f417-4b89-b2cb-fea836a3d57f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:19.989422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:20.012695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:20.013693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:20.013979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:20.014254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:20.015007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:20.016455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb409d0-30a0-46ea-ad71-0c184fb10c0a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:20.116913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 567/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:29:19 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:29:22.273578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c440-5b47-40f9-9f04-be53d1db86c7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:22.274105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c440-5b47-40f9-9f04-be53d1db86c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:22.274105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c440-5b47-40f9-9f04-be53d1db86c7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:22.275734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 568/1135] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:22.281360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:22.281815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:22.281930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:22.298713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:22.299920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:22.369041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 568/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:22.394291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36c440-5b47-40f9-9f04-be53d1db86c7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:22.420775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:22.505659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36c440-5b47-40f9-9f04-be53d1db86c7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:22.541420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:22.542224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:22.542502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:22.542805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:22.544005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:22.545439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8964df-1a90-4bf6-a05c-9fc32f858ebd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:22.547100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 569/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:22.553027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:22.653514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:22.659498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edb2f-aecf-47da-ab80-bbf7cdde01ef None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:22.705387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:22.705985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:22.706273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:22.706573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:22.707305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:22.708805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb92f0-d2af-4bc1-a835-bb9c390e9269 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:22.765637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 569/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:29:22 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:29:24.909173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695da7-81b6-4409-8d2b-2586ebfe1f41 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:24.909775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695da7-81b6-4409-8d2b-2586ebfe1f41 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:24.909775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4695da7-81b6-4409-8d2b-2586ebfe1f41 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:24.911104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 570/1139] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:24.917645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:24.918096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:24.918195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:24.935638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:24.936823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:25.004878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 570/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:25.029580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4695da7-81b6-4409-8d2b-2586ebfe1f41 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:25.127385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:25.137406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4695da7-81b6-4409-8d2b-2586ebfe1f41 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:25.170321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:25.171160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:25.171440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:25.171717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:25.183091 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:25.190795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:25.192026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:25.193314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 571/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:25.198833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:25.289456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:25.307951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d38a0-0489-4cb2-b7dd-e231598f7737 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:25.345446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:25.346111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:25.346392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:25.346699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:25.347454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:25.347758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:25.349134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:25.358810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 571/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:25.364979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:25.466418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4e06-078c-40ba-8ed7-7e6c5e885685 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:25.500997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:25.501879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:25.502153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:25.502424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] PATH_INFO: `/v3/users/4e726c320a86481d81770befafa82c9e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:25.519130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] RBAC: Authorizing `identity:get_user(user_id=4e726c320a86481d81770befafa82c9e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:25.520575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:25.522324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 572/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:25 2022] GET /identity/v3/users/4e726c320a86481d81770befafa82c9e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:25.529254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:25.634213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a45660-abf2-43eb-b73e-e7ffea2dba06 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:25.682487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:25.683162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:25.683441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:25.683713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:25.699658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:25.754861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:25.756204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:25.866288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 572/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 10:29:25.873457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:25.887123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:25.976692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681c838-a8c2-416e-8f7c-0dd240ef537e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:26.012308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:26.013083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:26.013368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:26.013648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] PATH_INFO: `/v3/projects/64327e76ae1c488a833270d6c40ec430` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:26.014859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] RBAC: Authorizing `identity:get_project(project_id=64327e76ae1c488a833270d6c40ec430)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:26.016302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22fcca-685b-42bc-b16e-a0355a692942 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:26.018018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 573/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:25 2022] GET /identity/v3/projects/64327e76ae1c488a833270d6c40ec430 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:26.027501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:26.074597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:26.129349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d89b9-ffc4-4d59-b6b5-ce77ac608539 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:26.163493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:26.164701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:26.165012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:26.165368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:26.166223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=64327e76ae1c488a833270d6c40ec430, user.id=4e726c320a86481d81770befafa82c9e) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:26.167068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:26.167999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a7c17-f084-41fe-9c51-0064f9f8f6fb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:26.174098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 573/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:26 2022] GET /identity/v3/role_assignments?user.id=4e726c320a86481d81770befafa82c9e&scope.project.id=64327e76ae1c488a833270d6c40ec430&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:28.234567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a5e6-8e32-46dd-8e89-79868079f0f6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.235259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a5e6-8e32-46dd-8e89-79868079f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.235259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a5e6-8e32-46dd-8e89-79868079f0f6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.236455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 574/1147] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:28.242071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.242499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.242748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.259610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:28.260811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:28.329755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 574/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 10:29:28.354242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23a5e6-8e32-46dd-8e89-79868079f0f6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:28.433647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:28.456191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23a5e6-8e32-46dd-8e89-79868079f0f6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:28.491273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:28.492069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.492345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.492773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.506120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:28.520543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:28.521680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:28.522767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 575/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:28.531567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:28.571069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:28.629632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd28a5-40d3-4e30-a010-3219c62f8db6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:28.663622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:28.664294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.664576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.664853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.665609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:28.665919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:28.667316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:28.675544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 575/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:28.682102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:28.784006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219973f2-dd84-4bd7-964f-4ad0c64d8cb3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:28.821578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:28.822442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.822750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.823026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] PATH_INFO: `/v3/users/4e726c320a86481d81770befafa82c9e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.824242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] RBAC: Authorizing `identity:get_user(user_id=4e726c320a86481d81770befafa82c9e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:28.825671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:28.827383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 576/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:28 2022] GET /identity/v3/users/4e726c320a86481d81770befafa82c9e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:28.833634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:28.933818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:28.936689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f040f-89db-4196-bec7-38d1f2ac241e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:28.971372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:28.972342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:28.972964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:28.973242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] PATH_INFO: `/v3/projects/64327e76ae1c488a833270d6c40ec430` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:28.974426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] RBAC: Authorizing `identity:get_project(project_id=64327e76ae1c488a833270d6c40ec430)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:28.975891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2367829-138f-4ac1-a079-f541502ec142 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:28.978305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 576/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:28 2022] GET /identity/v3/projects/64327e76ae1c488a833270d6c40ec430 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:28.984438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:28.993871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:29.084471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677dc37-b3f6-471c-ba69-91797b0b710a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:29.122985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:29.124373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:29.124790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:29.125200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] PATH_INFO: `/v3/projects/64327e76ae1c488a833270d6c40ec430/users/4e726c320a86481d81770befafa82c9e/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:29.128246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] RBAC: Authorizing `identity:create_grant(project_id=64327e76ae1c488a833270d6c40ec430, user_id=4e726c320a86481d81770befafa82c9e, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:29.130622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfa70c-69af-4caa-9308-ea0df45f5051 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:29.206505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 577/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:29:28 2022] PUT /identity/v3/projects/64327e76ae1c488a833270d6c40ec430/users/4e726c320a86481d81770befafa82c9e/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:29:31.244136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5db6-a6e1-4202-86cf-4fd5d88eef88 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.244679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5db6-a6e1-4202-86cf-4fd5d88eef88 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.244679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5db6-a6e1-4202-86cf-4fd5d88eef88 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:31.246298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 577/1154] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:29:31.251466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.251869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.252019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:31.269063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:31.270233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:31.329150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:31.336669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 578/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:31.361083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a5db6-a6e1-4202-86cf-4fd5d88eef88 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:31.462858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a5db6-a6e1-4202-86cf-4fd5d88eef88 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:31.496717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:31.497497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.497782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.498056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:31.508566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:31.516055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:31.517166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:31.518133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 578/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:31.524521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:31.624476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e3a3f-a95f-406b-bb04-51b6f68bb875 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:31.658438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:31.659129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.659450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.659723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:31.660456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:31.660755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:31.662100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:31.671547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 579/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:31.677537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:31.705128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:31.785080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf87c9-199e-4fdb-a999-9e77950fb489 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:31.821041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:31.821910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.822349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.822656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] PATH_INFO: `/v3/users/4e726c320a86481d81770befafa82c9e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:31.823876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] RBAC: Authorizing `identity:get_user(user_id=4e726c320a86481d81770befafa82c9e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:31.825313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:31.826983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 579/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:31 2022] GET /identity/v3/users/4e726c320a86481d81770befafa82c9e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:31.832886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:31.893847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:31.935359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecc515-4e16-4a9e-9f16-f6b4e9c00de8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:31.969697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:31.970348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:31.970650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:31.970924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:32.039336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:32.041155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:32.097743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:32.141012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 580/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:32.147008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:32.242733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:32.248956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b618b-6bef-4418-b5e2-ad715cb502ad None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:32.282253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:32.283137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:32.283419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:32.283690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] PATH_INFO: `/v3/projects/64327e76ae1c488a833270d6c40ec430` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:32.284874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] RBAC: Authorizing `identity:get_project(project_id=64327e76ae1c488a833270d6c40ec430)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:32.286303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efcf38-7c62-4bc7-878b-3f47e7ec77aa None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:32.288037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 580/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:32 2022] GET /identity/v3/projects/64327e76ae1c488a833270d6c40ec430 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:32.295202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:32.393307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ada3a-92cd-4c35-8269-e011c4bd16ee None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:32.427408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:32.428050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:32.428329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:32.428599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:32.430059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=64327e76ae1c488a833270d6c40ec430, user.id=4e726c320a86481d81770befafa82c9e) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:32.430364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:32.431819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270fe23-76b7-4737-b4e5-49834d8ebacb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:32.437753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 581/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:32 2022] GET /identity/v3/role_assignments?user.id=4e726c320a86481d81770befafa82c9e&scope.project.id=64327e76ae1c488a833270d6c40ec430&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:34.549513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cbce-d18d-408f-958b-e6f8a593c116 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:34.550044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cbce-d18d-408f-958b-e6f8a593c116 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:34.550044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cbce-d18d-408f-958b-e6f8a593c116 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:34.551400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 581/1162] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:34.557876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:34.558456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:34.558456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:34.575785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:34.576990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:34.594980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:34.652856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 582/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:34.676595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7cbce-d18d-408f-958b-e6f8a593c116 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:34.779070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7cbce-d18d-408f-958b-e6f8a593c116 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:34.813003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:34.813567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:34.813849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:34.814126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:34.844717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:34.881286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:34.883358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:34.979421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 582/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:34.985417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:35.084523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8417ee-9aa8-4510-857e-bcc1842252e7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:35.120384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:35.121194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:35.121479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:35.121754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:35.129199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:35.130687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d60db3b6-873b-4351-a7ed-70e519b52c34 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:35.145320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:35.146670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:35.147804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 583/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:29:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:35.153240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:35.217623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:35.256459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6bd3c-e3c3-4d66-a43a-106c46fda4c9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:35.290809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:35.291482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:35.291762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:35.292034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:35.292756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:35.293061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:35.294422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a4ec2-7597-4234-9dd1-e3ccf52c3adc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:35.301746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 583/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 21 10:29:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:37.348339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed658c9-9ceb-47d0-9816-0380c342dd35 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:37.348884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed658c9-9ceb-47d0-9816-0380c342dd35 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:37.348884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed658c9-9ceb-47d0-9816-0380c342dd35 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:37.350230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 584/1167] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:29:37.356608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0693d-4c44-445c-bab7-f60623bd2b0e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:37.357035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0693d-4c44-445c-bab7-f60623bd2b0e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:37.357158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0693d-4c44-445c-bab7-f60623bd2b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:37.374505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a6a0693d-4c44-445c-bab7-f60623bd2b0e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:37.375698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a6a0693d-4c44-445c-bab7-f60623bd2b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:37.444396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 584/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:37.467850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed658c9-9ceb-47d0-9816-0380c342dd35 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:37.484626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:37.571056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed658c9-9ceb-47d0-9816-0380c342dd35 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:37.605252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:37.605913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:37.606195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:37.606473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:37.607246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:37.608644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444d332-125b-4c14-a1f7-4ba5eb3d4ee9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:37.683446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 585/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Thu Jul 21 10:29:37 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:29:39.780229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750207-c8cb-4406-a5ea-397897f5ea2b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:39.780771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750207-c8cb-4406-a5ea-397897f5ea2b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:39.780771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750207-c8cb-4406-a5ea-397897f5ea2b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:39.782163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 585/1170] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:39.787678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:39.788161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:39.788877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:39.805665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:39.807244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:39.878923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 586/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:39.904317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750207-c8cb-4406-a5ea-397897f5ea2b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:39.965419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:40.012987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750207-c8cb-4406-a5ea-397897f5ea2b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:40.046415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:40.047213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:40.047492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:40.047766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] PATH_INFO: `/v3/domains/5a47847e80c64874aa7fcf13db876776` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:40.059142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a47847e80c64874aa7fcf13db876776)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:40.060599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d3c4d-bcd4-4517-aba3-2a4ad94b1feb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:40.062316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 586/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Jul 21 10:29:39 2022] GET /identity/v3/domains/5a47847e80c64874aa7fcf13db876776 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:40.068203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:40.070977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:40.176420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4c09-a35f-4b03-9a06-8ed96061e680 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:40.215478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:40.216136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:40.216416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:40.216691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:40.217407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:40.218869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2b7c6-d9c7-49db-ad4a-cafedb707094 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:40.299050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 587/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:29:40 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:29:42.428295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30640a6b-6776-4568-8a0e-85a66f81fb46 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:42.429343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30640a6b-6776-4568-8a0e-85a66f81fb46 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:42.429343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30640a6b-6776-4568-8a0e-85a66f81fb46 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:42.430827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 587/1174] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:42.436324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:42.436813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:42.436924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:42.454161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:42.455402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:42.512517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:42.530838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 588/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:42.556462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30640a6b-6776-4568-8a0e-85a66f81fb46 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:42.666144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30640a6b-6776-4568-8a0e-85a66f81fb46 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:42.684823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:42.702561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:42.703458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:42.703738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:42.704015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] PATH_INFO: `/v3/domains/5a47847e80c64874aa7fcf13db876776` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:42.705234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a47847e80c64874aa7fcf13db876776)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:42.706755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0502be-8658-43c2-b063-64f39707762c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:42.709365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 588/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Jul 21 10:29:42 2022] GET /identity/v3/domains/5a47847e80c64874aa7fcf13db876776 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:42.715151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:42.832958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc5e21-f940-4b08-92a5-29f40c50e8ef None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:42.868514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:42.869241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:42.869526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:42.870118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:42.870877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:42.872362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99247e-0850-4321-a23d-0b58dd0a56d8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:42.997253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 589/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:29:42 2022] POST /identity/v3/users => generated 311 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:29:45.183952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7c40-bf1b-4f6f-a362-2696fbecb028 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:45.184474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7c40-bf1b-4f6f-a362-2696fbecb028 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:45.184567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7c40-bf1b-4f6f-a362-2696fbecb028 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:45.185889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 589/1178] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:45.191438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:45.191955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:45.192050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:45.209700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:45.211523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:45.246793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:45.280636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 590/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:45.305249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a7c40-bf1b-4f6f-a362-2696fbecb028 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:45.411347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a7c40-bf1b-4f6f-a362-2696fbecb028 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:45.446200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:45.446932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:45.447219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:45.447497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:45.458932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:45.466844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:45.467991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:45.468891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 590/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:45.474222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:45.584692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f7b5-cb7c-480d-836a-c0d3c7eb1b89 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:45.625603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:45.630389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:45.641340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 591/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:45.647353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:45.747522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:45.806162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e967b-dab5-4c04-986c-f70bc8f2b18b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:45.842434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:45.843333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:45.843615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:45.843890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] PATH_INFO: `/v3/users/13be1a647f894f9aa86ef5c8cbe32933` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:45.857557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] RBAC: Authorizing `identity:get_user(user_id=13be1a647f894f9aa86ef5c8cbe32933)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:45.859095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:45.860841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 591/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:45 2022] GET /identity/v3/users/13be1a647f894f9aa86ef5c8cbe32933 => generated 311 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:45.867494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:45.901379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:45.973682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fffdf-f722-46ab-9064-9d753976b62a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:46.009812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:46.010375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:46.010685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:46.010961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:46.080491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:46.081924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:46.113695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:46.193352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 592/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:46.199598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:46.337636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c708-da8c-4c7e-85c7-8e2a18c8115b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:46.340684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:46.386266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:46.387171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:46.387521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:46.387719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] PATH_INFO: `/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:46.393597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] RBAC: Authorizing `identity:get_project(project_id=e0b205449a7c4e9084e8ee7d356a80d4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:46.393597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb40e10-32c0-4b48-ba36-52aec928c69f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:46.393597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 592/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:46 2022] GET /identity/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4 => generated 361 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:46.399680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:46.514912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc635bf3-a0fa-472a-a6fb-a6edf000c371 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:46.546922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:46.551717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:46.552412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:46.552692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:46.552964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:46.553803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=e0b205449a7c4e9084e8ee7d356a80d4, user.id=13be1a647f894f9aa86ef5c8cbe32933) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:46.554261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:46.556066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1e6f3-8651-4d25-9325-1b29b32e3a4f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:46.561830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 593/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:46 2022] GET /identity/v3/role_assignments?user.id=13be1a647f894f9aa86ef5c8cbe32933&scope.project.id=e0b205449a7c4e9084e8ee7d356a80d4&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:48.599776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cbd1c-0c0f-43a7-a59e-afc3aad3c9e3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:48.601032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cbd1c-0c0f-43a7-a59e-afc3aad3c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:48.601032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cbd1c-0c0f-43a7-a59e-afc3aad3c9e3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:48.602144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 593/1186] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:48.607421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:48.607861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:48.607954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:48.626567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:48.627766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:48.698012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 594/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:48.720861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cbd1c-0c0f-43a7-a59e-afc3aad3c9e3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:48.835211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cbd1c-0c0f-43a7-a59e-afc3aad3c9e3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:48.870764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:48.871472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:48.871752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:48.872027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:48.882604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:48.890315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:48.891445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:29:48.892317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 594/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:29:48.898278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:49.003445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee10252-7f7c-443b-a5b1-6161719eb4a7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:49.031936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:49.041369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:49.042044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:49.042326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:49.047042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:49.047042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:49.047042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:49.047042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:49.053853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 595/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:49.060247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:49.105372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:49.176394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1f27-15df-4295-bc59-c5c65447219a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:49.220278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:49.221150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:49.221433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:49.221705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] PATH_INFO: `/v3/users/13be1a647f894f9aa86ef5c8cbe32933` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:49.222928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] RBAC: Authorizing `identity:get_user(user_id=13be1a647f894f9aa86ef5c8cbe32933)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:49.224378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:49.226475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 595/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:49 2022] GET /identity/v3/users/13be1a647f894f9aa86ef5c8cbe32933 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:49.232513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:49.337021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9a6a-9430-4821-ba62-ad3b8c8053ac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:49.371377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:49.372227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:49.372506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:49.372784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] PATH_INFO: `/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:49.373957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] RBAC: Authorizing `identity:get_project(project_id=e0b205449a7c4e9084e8ee7d356a80d4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:49.375406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d490-478e-4edf-a1f7-5d00acb422fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:49.377093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 596/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:49 2022] GET /identity/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:49.383445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:49.490835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:49.490835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d2fdf-fd68-4a37-89da-5c7ff3f42b4d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:49.522084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:49.523294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:49.523576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:49.523849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] PATH_INFO: `/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4/users/13be1a647f894f9aa86ef5c8cbe32933/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:49.525977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0b205449a7c4e9084e8ee7d356a80d4, user_id=13be1a647f894f9aa86ef5c8cbe32933, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:49.527459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3832a-e24d-4b38-a6b1-38337bad22f6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:49.582764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 596/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:29:49 2022] PUT /identity/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4/users/13be1a647f894f9aa86ef5c8cbe32933/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:29:51.772095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8bf6b-eae0-4d03-aef1-b1818d471b7a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:51.772794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8bf6b-eae0-4d03-aef1-b1818d471b7a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:51.772794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8bf6b-eae0-4d03-aef1-b1818d471b7a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:51.774028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 597/1193] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:29:51.780168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:51.780702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:51.780702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:51.799863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:29:51.801084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:29:51.874517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 597/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:29:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:29:51.899822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8bf6b-eae0-4d03-aef1-b1818d471b7a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:51.965914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:52.015790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8bf6b-eae0-4d03-aef1-b1818d471b7a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:52.052668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:52.053384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:52.053733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:52.054009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:52.065690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:29:52.073809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:29:52.075078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:29:52.075951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 598/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:29:52.081453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:52.185829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:52.195937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c870a-8001-449b-aac2-6f1b303b35e5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:52.231381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:52.232066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:52.232347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:52.232624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:52.233387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:52.233696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:52.235113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:52.246207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 598/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:52.251704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:52.363298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5053d3-800d-47ea-b3b8-5a5174f8c563 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:52.380913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:52.400238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:52.401122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:52.401401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:52.401671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] PATH_INFO: `/v3/users/13be1a647f894f9aa86ef5c8cbe32933` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:52.402896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=13be1a647f894f9aa86ef5c8cbe32933)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:52.404366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:52.405995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 599/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 21 10:29:52 2022] GET /identity/v3/users/13be1a647f894f9aa86ef5c8cbe32933 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:29:52.412100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:52.532839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ece32-5075-4207-b863-8393e632c13f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:52.568017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:52.568682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:52.568967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:52.569816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:52.593888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:52.653355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:52.654773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:52.765604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 599/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:29:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:29:52.772446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:52.880926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a1b7d-ac31-4bbc-9ac7-a020b7c6b0f3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:52.917455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:52.918321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:52.918623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:52.918894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] PATH_INFO: `/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:52.920118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e0b205449a7c4e9084e8ee7d356a80d4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:52.921570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf2a3f-6fb5-465b-8a32-7c122a0445b9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:52.923412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 600/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 21 10:29:52 2022] GET /identity/v3/projects/e0b205449a7c4e9084e8ee7d356a80d4 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:29:52.930477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:29:52.985804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:29:53.036278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ee9b6-6dee-4df9-9446-695aa800f237 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:29:53.073947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:29:53.074649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:29:53.074936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:29:53.075209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:29:53.076056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=e0b205449a7c4e9084e8ee7d356a80d4, user.id=13be1a647f894f9aa86ef5c8cbe32933) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:29:53.076368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:29:53.077829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9808-b4ed-4a93-8964-76cbf909d3f1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:29:53.084602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 600/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:29:52 2022] GET /identity/v3/role_assignments?user.id=13be1a647f894f9aa86ef5c8cbe32933&scope.project.id=e0b205449a7c4e9084e8ee7d356a80d4&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:30:02.912233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=74 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:30:03.081515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=42 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:07.962869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142236-133c-4844-bc56-40a50c28d933 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:07.963510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142236-133c-4844-bc56-40a50c28d933 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:07.963510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142236-133c-4844-bc56-40a50c28d933 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:07.964803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 601/1201] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:07.970471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:07.970905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:07.971056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:07.987800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:07.988996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:08.060459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 601/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:08.085733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20142236-133c-4844-bc56-40a50c28d933 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:08.198707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20142236-133c-4844-bc56-40a50c28d933 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:08.233865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:08.234663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:08.234887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:08.235163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:08.246629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:08.248197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:08.266490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:08.274989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:08.276853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:08.276853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:08.276853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:08.276853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:08.276853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 602/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:08.277459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:08.372537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899442a-ea14-4fee-bab3-25cdaad5fd66 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:08.401210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:08.406971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:08.407618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:08.407899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:08.408173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:08.408889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:08.409223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:08.410602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ea37b-da2d-4583-a76a-2627f5969cfe None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:08.422295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 602/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:08.429205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:08.482786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:08.528893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ee9cf-641d-48f4-ace1-30088ae9ff20 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:08.561761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:08.562419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:08.562731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:08.563011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:08.563716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:08.565131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e79bb7-ac0f-4ade-8175-aebac62de45a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:08.764088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 603/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 21 10:31:08 2022] POST /identity/v3/users => generated 251 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:31:10.862245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf5d2-f670-463c-8d9b-3c44246b2869 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:10.862847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf5d2-f670-463c-8d9b-3c44246b2869 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:10.862847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf5d2-f670-463c-8d9b-3c44246b2869 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:10.864251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 603/1206] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:10.870987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:10.871500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:10.871612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:10.888303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:10.889539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:10.962688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 604/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:10.988011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf5d2-f670-463c-8d9b-3c44246b2869 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.100792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf5d2-f670-463c-8d9b-3c44246b2869 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:11.138012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:11.138765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:11.139044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:11.139316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:11.150366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:11.158078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:11.159262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:11.160297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 604/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:11.165667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.208978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:11.268932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db787897-737a-4806-ab76-6f8d642b1d01 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:11.304356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:11.305838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:11.306124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:11.306408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:11.307190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:11.307496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:11.308866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:11.317344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 605/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:11.324526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.379325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:11.430314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf4e4-1375-4a76-8c85-a074a4029074 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:11.470893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:11.471664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:11.471938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:11.472205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:11.483678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:11.485133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:11.505457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:11.508483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:11.509501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:11.509501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:11.509501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:11.509501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:11.509501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 605/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:11.513150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.612369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:11.615355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd75cd-f241-49ef-ba72-29d0a62cb3d1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:11.651120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:11.651784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:11.652067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:11.652346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:11.653054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:11.653488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:11.654877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:11.667349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 606/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:11.673498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.788792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef85f7-ad14-4f4b-aacd-9edec0d85a24 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:11.791131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:11.825382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:11.826213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:11.826494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:11.826800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:11.835824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:11.837843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:11.853795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:11.855314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:11.856665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 606/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:11.861883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:11.981490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cd058-14ea-4baa-8a0d-7ba52d7fcccc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:12.018078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:12.018862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:12.019146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:12.019421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:12.020246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:12.020553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:12.021970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:12.044873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 607/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:12.046112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:12.145081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518af9d-a643-4f2c-8c95-b8ea07b44a33 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:12.178800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:12.179453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:12.179797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:12.180249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:12.209941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:12.246986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:12.248559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:12.351261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 607/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:31:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:31:12.358403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:12.380845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:12.485749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efab2d6-f2d8-40e9-a5f0-3a1599a8237d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:12.533701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:12.534487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:12.534800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:12.535074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:12.551100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:12.552729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:12.579242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:12.581775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:12.583637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:12.583637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:12.583637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:12.583637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:12.583637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 608/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:12.586861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:12.661336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:12.705411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b315b3-f88b-4ac0-a065-e46f71c41cab None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:12.757361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:12.759257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:12.759544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:12.759815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:12.760548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:12.760863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:12.762237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:12.781744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 608/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:12.788385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:12.885823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:12.918816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f81c5-b6fa-4687-80ef-699aab83ac1c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:12.958968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:12.960397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:12.960675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:12.961450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:12.971817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:12.973249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:12.991004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:12.993607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:12.994901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 609/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:13.000106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:13.116255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa51f63-61a7-45da-821e-55f0b3681a3c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:13.143886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:13.157178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:13.157860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:13.158138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:13.158412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:13.159242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:13.159551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:13.161140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f15188-0772-4289-b9e3-b8aa088fa544 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:13.174900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 609/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:13.181859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:13.293821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd102a-2f8c-4e68-bdbf-0e28a50a55d4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:13.324299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:13.330160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:13.330848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:13.331132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:13.331408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:13.332235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=f4d6a7f8781c4139a4d927dc47cf1a36) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:13.332546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:13.334020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22cba9-2063-4e4a-a419-ac2141b0e991 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:13.340525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 610/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:31:13 2022] GET /identity/v3/role_assignments?user.id=f4d6a7f8781c4139a4d927dc47cf1a36&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:15.459513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a86d8-fd90-4eb8-bbe5-33fc753c7c7b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:15.460064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a86d8-fd90-4eb8-bbe5-33fc753c7c7b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:15.460064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a86d8-fd90-4eb8-bbe5-33fc753c7c7b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:15.462081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 610/1220] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:15.467594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:15.468003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:15.468144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:15.485675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:15.486948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:15.553409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 611/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:15.577180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a86d8-fd90-4eb8-bbe5-33fc753c7c7b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:15.679206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a86d8-fd90-4eb8-bbe5-33fc753c7c7b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:15.714321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:15.715058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:15.715335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:15.718624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:15.725460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:15.733039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:15.734171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:15.735274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 611/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:31:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:15.740209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:15.839176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc252c-0551-4273-95cd-f6e0977d96a0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:15.862166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:15.873288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:15.873955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:15.874239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:15.874518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:15.875294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:15.875604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:15.876992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:15.885377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 612/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:31:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:15.891464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:15.936052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:15.990954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd7e69-2372-4497-a417-7c6fe79db014 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.024594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.025381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.025663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.025954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.036485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.037929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:16.056727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:16.060212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:16.061333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:16.061333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:16.061333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:16.061333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:16.061333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 612/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:16.064013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.164559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6ca0b-d347-467d-bd75-877a2758b832 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.199460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.200115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.200394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.200664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.201565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:16.201863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.203237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.215403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 613/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:16.221467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.315949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:16.322367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710bb7-7779-43d5-9a9b-766655add855 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.354880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.355679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.355958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.356227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.363916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.365458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:16.380348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:16.381482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:16.382578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 613/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:16.387900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.436490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:16.490308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af7dc7-b8de-488c-9cfa-29719c339eab None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.526287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.527000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.527321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.527601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.528423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:16.528729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.530153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.548249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 614/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:16.553994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.655056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a504b-6143-4010-adba-6d891af5fddc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.689590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.690374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.690688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.690969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.701957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.703822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:16.720313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:16.721434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:16.732873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:16.732873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:16.732873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:16.732873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:16.732873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 614/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:16.733260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.825725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f340c4-2628-45fc-ab49-66a9041356f9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:16.828111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:16.858697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:16.859541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:16.860051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:16.860388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:16.861416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:16.861760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:16.863213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:16.875460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 615/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:16.881204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:16.891047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:16.981383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b7bb7-ccf2-44da-a5d4-0575d37afa6c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:17.014391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:17.015198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:17.015549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:17.015752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:17.022287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:17.024403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:17.038247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:17.040240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:17.041430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 615/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:17.046115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:17.149518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee194c19-6030-43a5-b511-df2711a022f5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:17.186350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:17.187040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:17.187313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:17.187584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:17.188360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:17.188666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:17.190100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24089d6c-ca8c-4c8a-8df4-4259caa2b3e8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:17.202836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 616/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:17.208825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:17.277332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:17.312860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456533e-2f9f-41c1-a34a-d97afda1bfb8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:17.348849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:17.349755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:17.350032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:17.350341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/f4d6a7f8781c4139a4d927dc47cf1a36/roles/73e8adf6da3c4f09a86a7a5ca94398b7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:17.365996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=f4d6a7f8781c4139a4d927dc47cf1a36, role_id=73e8adf6da3c4f09a86a7a5ca94398b7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:17.368902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ef939-764f-4881-9f85-e827336c8768 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:17.454930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 616/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:31:17 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/f4d6a7f8781c4139a4d927dc47cf1a36/roles/73e8adf6da3c4f09a86a7a5ca94398b7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:31:19.521371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bf54-2f55-47bd-871c-745f9afdc1a6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:19.521885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bf54-2f55-47bd-871c-745f9afdc1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:19.521995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bf54-2f55-47bd-871c-745f9afdc1a6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:19.523962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 617/1233] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:19.529915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:19.530413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:19.534594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:19.547987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:19.549181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:19.620498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 617/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:19.645132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bf54-2f55-47bd-871c-745f9afdc1a6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:19.672047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:19.751765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bf54-2f55-47bd-871c-745f9afdc1a6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:19.789572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:19.790276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:19.790585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:19.790860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:19.802425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:19.810910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:19.812147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:19.813661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 618/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 21 10:31:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:19.818287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:19.871874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:19.924371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6fc9-9a3a-464a-87af-3e8b0f8bde41 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:19.956903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:19.957571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:19.957855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:19.958131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:19.958913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:19.959223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:19.960595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:19.969440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 618/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 21 10:31:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:19.975199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:20.063216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:20.079223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1f3f5-be81-4381-950a-7e6400fe77a5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:20.113973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:20.114794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:20.115081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:20.115360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:20.126465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:20.127949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:20.146039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:20.147176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:20.149212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:20.149212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:20.149212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:20.149212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:20.149212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 619/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:20.154091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:20.255567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e99bb-4813-41bf-a1f8-275a8884743a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:20.268401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:20.295623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:20.296294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:20.296577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:20.296852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:20.297581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:20.297888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:20.299309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:20.310812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 619/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:20.317329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:20.425997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42ae-c3a9-47ce-8d02-97922db310e7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:20.461914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:20.462783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:20.463064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:20.463336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:20.466040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:20.470920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:20.472350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:20.487663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:20.488855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:20.495286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 620/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:20.496026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:20.599248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0335c8-5f71-4249-944d-75fd0576b3f9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:20.635657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:20.636334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:20.636615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:20.642029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:20.642029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:20.642029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:20.642029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:20.657745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 620/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:20.663634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:20.767261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3416ce-f21e-4a3b-b0c0-019227ca08fc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:20.811212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:20.811860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:20.812179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:20.812558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:20.876911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:20.883723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:20.885261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:20.983393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 621/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:31:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:31:20.989501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:21.017397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:21.089809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c78b6-a92c-4fa2-b890-3c1fc34695d9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:21.134643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:21.135470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:21.135780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:21.136049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:21.146384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:21.147871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:21.166203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:21.167262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:21.175621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:21.175621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:21.175621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:21.175621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:21.175621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 621/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:31:21.175974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:21.269897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:21.277849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07e89-1f0a-4a00-a3b4-5f8cff8ccba7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:21.314061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:21.314807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:21.315041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:21.315319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:21.316063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:21.316642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:21.319368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:21.332311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 622/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:21.339188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:21.420853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:21.443039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d6b9d-bb3c-4f4a-a7d5-0f9748bdc61c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:21.482565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:21.483387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:21.483668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:21.483937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:21.490984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:21.493577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:21.508164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:21.511713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:21.512820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 622/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:21.516975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:21.628346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f4356-d2d6-44ed-87b3-d2022f329a1a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:21.667055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:21.667756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:21.667980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:21.668254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:21.669027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:21.669332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:21.670799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e730dd8-2ad5-425a-9fce-bf7a3838a43a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:21.689088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 623/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:21.691305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:21.800209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c63fb-8db9-43ac-9543-baac783a7da9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:21.835001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:21.836195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:21.836480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:21.836756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:21.837780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] RBAC: Adding query filter params (role.id=73e8adf6da3c4f09a86a7a5ca94398b7, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=f4d6a7f8781c4139a4d927dc47cf1a36) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:21.838136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:21.839594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8c8b9-4002-4393-bc41-baeb6a531333 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:21.853853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 623/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:31:21 2022] GET /identity/v3/role_assignments?user.id=f4d6a7f8781c4139a4d927dc47cf1a36&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=73e8adf6da3c4f09a86a7a5ca94398b7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:24.015431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561ed7-a59e-4acb-b642-4f24f434274b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.015999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561ed7-a59e-4acb-b642-4f24f434274b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.015999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561ed7-a59e-4acb-b642-4f24f434274b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.017385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 624/1247] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:24.022773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.023234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.023579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.028773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:24.041034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:24.042315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:24.125460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 624/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:24.149236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561ed7-a59e-4acb-b642-4f24f434274b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:24.168469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:24.257342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561ed7-a59e-4acb-b642-4f24f434274b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:24.292342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:24.293052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.293335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.293616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.304063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:24.312263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:24.313389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:24.314256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 625/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:31:24.319577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:24.423653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8542d-3df4-469a-88e6-4cb5e223942e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:24.457227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:24.462796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:24.463465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.463746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.464020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.464829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:24.465184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:24.466500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:24.474901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 625/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:24.482226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:24.570537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:24.594339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7badf3d-6290-407a-bc60-b6ecde977f9b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:24.630061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:24.630958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.631276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.631588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.645282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:24.657485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:24.667512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:24.669064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:24.670439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:24.670439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:24.670439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:24.670439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:24.670439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 626/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:24.675489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:24.790122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097e54a-2067-46ab-bcae-69e4c9bbaaea None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:24.827046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:24.827646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.827927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.828206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:24.828933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:24.829237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:24.830731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:24.843983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 626/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:24.850033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:24.955975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956972ff-d820-45cc-b16a-6389ef609525 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:24.984244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:24.991808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:24.993128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:24.993407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:24.993681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:25.001251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:25.002724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:25.018036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:25.024732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:25.026323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 627/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:25.027156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:25.095247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:25.137994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adef1-43ca-46a4-ad95-4939547172ef None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:25.175344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:25.176018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:25.176300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:25.176575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:25.177411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:25.177720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:25.179204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:25.196968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 627/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:25.202809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:25.303645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aab6ba-0855-4dd0-87a0-8d60be2ae12a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:25.335129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:25.336310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:25.336634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:25.336909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:25.391878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:25.403796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:25.405222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:25.501250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 628/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:31:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:31:25.507096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:25.609580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420615-90f4-45ce-bcb2-c1c1810b21e3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:25.623755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:25.646818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:25.647660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:25.647942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:25.648361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:25.659090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:25.666048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:25.681584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:25.689724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:25.690863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:25.690863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:25.690863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:25.690863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:25.690863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 628/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:25.691192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:25.782741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:25.797234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb693-0e0d-42be-a793-843fefa5b1bf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:25.834643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:25.836080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:25.836532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:25.836963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:25.838148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:25.838676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:25.841045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:25.860990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 629/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:25.866557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:25.975218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964f7fa-0e39-44f6-b290-0009430a7ef2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:26.012478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:26.013273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:26.013586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:26.013831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:26.020622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:26.022062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:26.036399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:26.038033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:26.039933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 629/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:26.044340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:26.148786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffce47c-c861-4214-a2b6-46767714add7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:26.184281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:26.184943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:26.185221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:26.185493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:26.186301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:26.186667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:26.188136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902e4fc-b0a4-4309-8048-41365141e890 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:26.201140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 630/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:26.207391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:26.239973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:26.313021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b9b19-c4cf-4662-b126-f14d1d0e664d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:26.349360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:26.350020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:26.350311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:26.350609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:26.352282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=f4d6a7f8781c4139a4d927dc47cf1a36) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:26.352600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:26.354035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0490326-6cc9-48dd-9d11-b0df47297d24 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:26.359911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 630/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:31:26 2022] GET /identity/v3/role_assignments?user.id=f4d6a7f8781c4139a4d927dc47cf1a36&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:28.436340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff24b1-899e-460c-8384-96ae6a70f712 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:28.436905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff24b1-899e-460c-8384-96ae6a70f712 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:28.436905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff24b1-899e-460c-8384-96ae6a70f712 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:28.438285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 631/1261] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:28.444111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:28.444640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:28.444640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:28.462512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:28.463799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:28.532531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 631/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:28.556067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff24b1-899e-460c-8384-96ae6a70f712 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:28.567329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:28.666764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff24b1-899e-460c-8384-96ae6a70f712 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:28.701516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:28.702289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:28.702861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:28.703040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:28.714182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:28.721904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:28.723154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:28.724294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 632/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:28.729963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:28.751130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:28.838864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220f27-e883-4376-b85e-171a3ee57ee4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:28.877522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:28.878193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:28.878474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:28.878780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:28.879560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:28.879875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:28.881293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:28.889724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 632/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:28.895814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:28.967867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:29.002446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daffdac-73ca-4e7e-8006-988e31facce4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.036335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.037136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.037415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.037742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.049560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.051052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:29.069272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:29.073926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:29.075648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:29.075648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:29.075648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:29.075648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:29.075648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 633/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:29.077787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.161977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:29.183335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758fdf-2414-4067-b32e-35e937906cd0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.221124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.221839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.222119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.222397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.223154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:29.223460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.224870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.241634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 633/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:29.244105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.353297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d455d5-351c-4fc5-9890-b4ac7ba4c9d2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.380132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:29.395242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.396045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.396326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.396600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.404484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.405909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:29.420883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:29.427862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:29.429051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 634/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:29.430438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.552201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad062f5-8eaf-4fcb-b754-749e18ba6e0f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.588911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.589608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.589924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.590299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.591174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:29.591479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.592914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.602840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:29.611759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 634/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:29.619142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.722055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d59bc-1f76-4096-ac41-425517ceff59 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.757450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.758313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.758617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.758951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.769698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.776339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:29.790714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:29.794313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:29.795508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:29.795508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:29.795508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:29.795508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:29.795508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 635/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:29.797770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.903488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99830b26-76de-497d-b721-95ff54b41b7e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:29.938845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:29.939514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:29.939799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:29.940078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:29.940806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:29.941120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:29.942498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:29.954938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 635/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:29.961271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:29.995540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:30.065405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986d38-bc8a-4f72-aec5-1343949834ba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:30.101764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:30.102631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:30.102958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:30.103237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:30.111533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:30.114068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:30.127761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:30.128876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:30.130227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 636/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:31:30.135928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:30.199284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:30.244767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771d0db-fb9a-432f-ab13-fab46dc91a7e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:30.279513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:30.280181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:30.280464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:30.280737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:30.281574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:30.281913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:30.283410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c045f-8336-4182-8560-02305d537a0c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:30.296503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 636/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:30.302558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:30.397597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:30.406415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a501463-ef3d-4b0b-9cce-b528bd1323cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:30.441861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:30.442699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:30.442985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:30.443267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] PATH_INFO: `/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/f4d6a7f8781c4139a4d927dc47cf1a36/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:30.445254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] RBAC: Authorizing `identity:create_grant(project_id=984cdf29aa8f4164bd840cd10a9902fe, user_id=f4d6a7f8781c4139a4d927dc47cf1a36, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:30.446754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddb81c-6c9e-4781-ac44-198c010d6408 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:30.620682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 637/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 21 10:31:30 2022] PUT /identity/v3/projects/984cdf29aa8f4164bd840cd10a9902fe/users/f4d6a7f8781c4139a4d927dc47cf1a36/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:31:32.687098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b82e7-0d63-42dc-989b-2dbc4167d31f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:32.687723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b82e7-0d63-42dc-989b-2dbc4167d31f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:32.687723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b82e7-0d63-42dc-989b-2dbc4167d31f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:32.689025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 637/1274] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:32.694433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:32.694935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:32.695111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:32.712476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:32.713658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:32.782968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 638/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:32.808388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b82e7-0d63-42dc-989b-2dbc4167d31f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:32.914845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b82e7-0d63-42dc-989b-2dbc4167d31f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:32.932769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:32.950006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:32.950765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:32.951048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:32.951321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:32.961827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 10:31:32.969722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:32.970932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:32.972027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 638/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 21 10:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:32.978073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.077364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:33.098474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc961b7-9ede-4c9b-9bb7-d840f9975aa5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:33.138425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:33.139171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:33.139450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:33.139724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:33.140475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:33.140779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:33.142176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:33.151775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 639/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 21 10:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:33.158569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.274901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f854040-9eef-4e33-9849-70dd0c0cbcdc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:33.312714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:33.313509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:33.313787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:33.314049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:33.328011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:33.329497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:33.347182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:33.348154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:33.349068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:33.349068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:33.349068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:33.349068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:33.349068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 639/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:33.354985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.469208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9574ee-f3fb-4d9e-a86a-e092809d4b9b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:33.507962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:33.508613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:33.508891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:33.509165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:33.509870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:33.510171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:33.511550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:33.524080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 640/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:31:33.530018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.562801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:33.638011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7cea-9963-4a78-a80d-b74c8ff9486f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:33.673359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:33.674186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:33.674464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:33.674780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:33.683701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:33.685142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 10:31:33.700794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:33.708932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:33.709903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 640/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:33.711076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.712428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:33.811996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ff545-b1a0-42af-8125-ef4f9ee15d72 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:33.846744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:33.847414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:33.847693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:33.847968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:33.848797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:33.849120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:33.850525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:33.869543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 641/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 21 10:31:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:33.875488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:33.970849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:33.980411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9bd34-a5fa-4089-99a2-d8ad1920bfb8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:34.015555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:34.016198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:34.016479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:34.016753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:34.085403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:34.087152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:34.183670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:34.196737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 641/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 21 10:31:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 10:31:34.203783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:34.267541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:34.307515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f10a-e674-49ac-af73-f4844ea4c995 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:34.343507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:34.344222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:34.344508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:34.344782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:34.357672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:34.359213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 10:31:34.376850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:34.377998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:34.379141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 10:31:34.379141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 10:31:34.379141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 10:31:34.379141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:34.379141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 642/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:31:34.386608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:34.488301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3929b-46b6-4647-b871-a34f42bd55f1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:34.524461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:34.525785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:34.526069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:34.526350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:34.527111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:34.527422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:34.528795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:34.541454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 642/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 21 10:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:34.547267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:34.652727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9ae8e-b115-44d1-a79f-03e26cdaa6cb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:34.667187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:34.690420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da269e-b437-4104-b703-0c345291d97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:34.691254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da269e-b437-4104-b703-0c345291d97d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:34.691536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da269e-b437-4104-b703-0c345291d97d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:34.691811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da269e-b437-4104-b703-0c345291d97d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:34.699353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da269e-b437-4104-b703-0c345291d97d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:34.700824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da269e-b437-4104-b703-0c345291d97d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-85da269e-b437-4104-b703-0c345291d97d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 10:31:34.716958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:34.719122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jul 21 10:31:34.720888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 643/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 21 10:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:34.724315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:34.781731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:34.833213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7011897-8510-4c70-af3c-01ec3a0c5c03 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:34.871476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:34.872141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:34.872424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:34.872699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:34.873479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:34.873788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:34.875293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c135e4-95c0-4733-8388-07e68196a2d2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:34.888898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 643/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 21 10:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:34.895161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da269e-b437-4104-b703-0c345291d97d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:34.996162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da269e-b437-4104-b703-0c345291d97d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:35.033713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:35.034368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:35.034679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:35.034983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:35.035810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] RBAC: Adding query filter params (role.id=de1cc43c43db46bfba3665cbe83fd0c6, scope.project.id=984cdf29aa8f4164bd840cd10a9902fe, user.id=f4d6a7f8781c4139a4d927dc47cf1a36) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:35.036122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:35.037569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1e17a-2fda-4422-9f35-e087580ccd18 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:35.043919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 644/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 21 10:31:34 2022] GET /identity/v3/role_assignments?user.id=f4d6a7f8781c4139a4d927dc47cf1a36&scope.project.id=984cdf29aa8f4164bd840cd10a9902fe&role.id=de1cc43c43db46bfba3665cbe83fd0c6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:37.175183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5402e8d-8fa2-4827-b3ab-07f8424bbbd5 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:37.175183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5402e8d-8fa2-4827-b3ab-07f8424bbbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:37.175183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5402e8d-8fa2-4827-b3ab-07f8424bbbd5 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:37.175183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 644/1288] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:37.179294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:37.179754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:37.179874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:37.197805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:37.199031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:37.229682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:37.269154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 645/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:37.292538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5402e8d-8fa2-4827-b3ab-07f8424bbbd5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:37.404435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5402e8d-8fa2-4827-b3ab-07f8424bbbd5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:37.415800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:37.440485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:37.441290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:37.441571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:37.441846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:37.442499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:37.443899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:37.456650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:37.457968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:37.461867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 645/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:31:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:37.464011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:37.573420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba93f4-f919-48f8-bf2a-39aef6e8fc89 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:37.608023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:37.608697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:37.609114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:37.609392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:37.610097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:37.610424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:37.611814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fc071-606b-4f39-8a9d-4d07ffd58255 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:37.617078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 646/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 21 10:31:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:37.623433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:37.725670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aefa7a-3423-4e9a-b599-f991807d943b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:37.760214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:37.760930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:37.761212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:37.761486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:37.762198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:37.762503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:37.763900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b07a37-68ae-42a0-aa54-f02df0ef058b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:37.768497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 646/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 21 10:31:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:39.874316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e27c1-bc6e-44f5-b4c4-7072d304b5dc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:39.875259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e27c1-bc6e-44f5-b4c4-7072d304b5dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:39.875427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e27c1-bc6e-44f5-b4c4-7072d304b5dc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:39.876855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 647/1293] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:39.882827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd6f-03b4-4e12-88b2-b54f5071efd1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:39.883319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd6f-03b4-4e12-88b2-b54f5071efd1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:39.883430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd6f-03b4-4e12-88b2-b54f5071efd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:39.904038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b1c6fd6f-03b4-4e12-88b2-b54f5071efd1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:39.914424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b1c6fd6f-03b4-4e12-88b2-b54f5071efd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:39.934571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:39.977172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 647/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:40.002100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e27c1-bc6e-44f5-b4c4-7072d304b5dc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:40.034794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:40.116355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e27c1-bc6e-44f5-b4c4-7072d304b5dc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:40.153524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:40.154096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:40.154372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:40.154710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:40.155415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:40.156762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f7f0b-4cd1-4596-b478-afcd7055b54e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:40.285527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 648/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 21 10:31:40 2022] POST /identity/v3/services => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:31:42.378094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe726-3477-45f5-b0bf-6f72e6dc60cc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:42.378760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe726-3477-45f5-b0bf-6f72e6dc60cc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:42.378760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe726-3477-45f5-b0bf-6f72e6dc60cc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:42.380080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 648/1296] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:42.385936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:42.386448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:42.386570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:42.404221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:42.405470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:42.485813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 649/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:42.509213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe726-3477-45f5-b0bf-6f72e6dc60cc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:42.617749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe726-3477-45f5-b0bf-6f72e6dc60cc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:42.654020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:42.654852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:42.655137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:42.655410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:42.656072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:42.657446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:42.675765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:42.677078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:42.678177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 649/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:31:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 10:31:42.683875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:42.706629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:42.792880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021a7ce-0c87-400b-925b-7d92ef9e6aaf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:42.830004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:42.830700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:42.830986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:42.831261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:42.831972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:42.832280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:42.836610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:42.840487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 650/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 21 10:31:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:42.847305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:42.881236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:42.955749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b3d02-58c5-49b4-a914-ac0cc32ffc83 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:42.994359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:42.995067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:42.995343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:42.995614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:42.996409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] RBAC: Adding query filter params (interface=public, service_id=381916c7e47543a1871a833513a120ac, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:42.996715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:42.998140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4231eb-5f38-47a0-93be-2fdd9ea596a5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:43.003909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 650/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 21 10:31:42 2022] GET /identity/v3/endpoints?service_id=381916c7e47543a1871a833513a120ac&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:43.009918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:43.093276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:43.113914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b3e42-9f70-45e9-a52d-5e6b964b6e4a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:43.150120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:43.150820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:43.151104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:43.151376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:43.152048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:43.153421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d8181-c6e6-4745-b590-f0ee23e8bbe5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:43.160590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 651/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 21 10:31:43 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:31:45.235808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321dd-991d-4e3b-9706-e22649a2e6c1 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:45.236367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321dd-991d-4e3b-9706-e22649a2e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:45.236481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a321dd-991d-4e3b-9706-e22649a2e6c1 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:45.237885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 651/1302] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:31:45.243776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:45.247448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:45.247448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:45.263291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:31:45.264495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:31:45.336569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 652/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:31:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 10:31:45.361731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a321dd-991d-4e3b-9706-e22649a2e6c1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:45.463593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:45.470527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a321dd-991d-4e3b-9706-e22649a2e6c1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:45.505290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:45.506232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:45.506511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:45.506817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:45.507506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:45.508921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 10:31:45.523104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 10:31:45.535147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Jul 21 10:31:45.536363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 652/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 21 10:31:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 10:31:45.537518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:45.613024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:45.638072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2893d82-a6b4-432a-bf71-889c4008fbff None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:45.672279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:45.672940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:45.673223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:45.673498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:45.674194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:31:45.674499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:45.675961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437336-4451-43de-962f-5a306ae1da4e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:45.683041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 653/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 21 10:31:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:31:45.687801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:31:45.789830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ba4e-b088-4851-802a-eecfeb2a9f9b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:31:45.825104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:31:45.825765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:31:45.826043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:31:45.826339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:31:45.827101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:31:45.828481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54edf66-c693-4134-a7d7-386a2c89b7f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:31:45.911269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 653/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 21 10:31:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:31:55.683347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=30 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:31:55.906336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:24.871780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2797-1137-490c-8e7f-baed72399ac0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:24.872798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2797-1137-490c-8e7f-baed72399ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:24.873294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2797-1137-490c-8e7f-baed72399ac0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:24.880938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fb0b4-3dad-49ae-8b63-3d2a34a8e19f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:24.881852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fb0b4-3dad-49ae-8b63-3d2a34a8e19f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:24.882394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fb0b4-3dad-49ae-8b63-3d2a34a8e19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:24.886295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 654/1307] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:24 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:25.017142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b68fb0b4-3dad-49ae-8b63-3d2a34a8e19f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:25.022199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b68fb0b4-3dad-49ae-8b63-3d2a34a8e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:25.207199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 654/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:37:25.265641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:25.266005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:25.266399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:25.268547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 655/1309] 10.222.0.48 () {54 vars in 965 bytes} [Thu Jul 21 10:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:25.275282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f477b-3405-4640-a6b1-76cb601d42d1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:25.275590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f477b-3405-4640-a6b1-76cb601d42d1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:25.275936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f477b-3405-4640-a6b1-76cb601d42d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:25.374288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3a9f477b-3405-4640-a6b1-76cb601d42d1 None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:25.375460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3a9f477b-3405-4640-a6b1-76cb601d42d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:25.403568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:25.476160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 655/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Jul 21 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:25.486824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:25.488631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:25.622173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadfceb-5b1d-4533-889b-f195ca4a68f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:25.707312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:25.707888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:25.708233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:25.708443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:25.710122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:25.771122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:25.832312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:25.833674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5d850-6445-4935-ab5a-ede37a63e7b2 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:25.854279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 656/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:28.289704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04ee1-ce00-4213-9df4-8813b318b44a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:28.290171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04ee1-ce00-4213-9df4-8813b318b44a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:28.290171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04ee1-ce00-4213-9df4-8813b318b44a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:28.291712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 656/1312] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:28.297176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfd5-967c-45b9-9fd8-0be0846c81dc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:28.297590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfd5-967c-45b9-9fd8-0be0846c81dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:28.297776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfd5-967c-45b9-9fd8-0be0846c81dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:28.392554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1a90dfd5-967c-45b9-9fd8-0be0846c81dc None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:28.393907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1a90dfd5-967c-45b9-9fd8-0be0846c81dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:28.499661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 657/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:28.552461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:28.552748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:28.553026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:28.554692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 657/1314] 10.222.0.48 () {54 vars in 965 bytes} [Thu Jul 21 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:37:28.560427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5bc9-cdc9-4ffb-a3e2-6956a0b5a47f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:28.560786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5bc9-cdc9-4ffb-a3e2-6956a0b5a47f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:28.561135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5bc9-cdc9-4ffb-a3e2-6956a0b5a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:28.572540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:28.643799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4ddc5bc9-cdc9-4ffb-a3e2-6956a0b5a47f None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:28.645021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ddc5bc9-cdc9-4ffb-a3e2-6956a0b5a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:28.725227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 658/1315] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Jul 21 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:28.734323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:28.736490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:28.845745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72693f74-77b7-4ed3-b980-e7eb234a6458 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:28.922383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:28.926324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:28.927065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:28.927387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:28.927716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:28.929595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:29.056602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:29.058253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3dc9e1-c535-4be8-9055-668438ceea22 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:29.079232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 658/1316] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:37:31.973670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:31.974214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:31.974214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:31.975684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 659/1317] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:31.981102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858accf-e9c2-464b-a106-29ad4ee8d0a6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:31.981589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858accf-e9c2-464b-a106-29ad4ee8d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:31.981686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858accf-e9c2-464b-a106-29ad4ee8d0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:32.044598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6858accf-e9c2-464b-a106-29ad4ee8d0a6 None None] MFA Rules not processed for user `ed5bda805a18497ab72e318c7b4698e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:32.045787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6858accf-e9c2-464b-a106-29ad4ee8d0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:32.097948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:32.142970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 659/1318] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:32.183689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:32.185221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:32.293141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a41cbf-588f-46ec-8307-f4c3942faf5e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:32.295976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:32.369208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:32.369858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:32.370140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:32.370415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:32.372106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:32.481195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:32.482683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6507e2c-8e4a-4359-bf8a-55fc647d05f5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:32.502262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 660/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:35.394439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:35.395131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:35.395313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:35.396537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 660/1320] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:35.401994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8fed-59ea-4c10-b587-7e7764fbb4f9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:35.402413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8fed-59ea-4c10-b587-7e7764fbb4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:35.402567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8fed-59ea-4c10-b587-7e7764fbb4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:35.443184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:35.468227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d52f8fed-59ea-4c10-b587-7e7764fbb4f9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:35.469434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d52f8fed-59ea-4c10-b587-7e7764fbb4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:35.569853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 661/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:35.617757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:35.619367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:35.726239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d076f8-be26-49c5-a7ea-f31e3c2e63bd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:35.800463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:35.801110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:35.801390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:35.801666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:35.803361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:35.810830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:35.914153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:35.915643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517469-bfeb-4f42-8d34-20b2ef70fcc7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:35.934070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 661/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:39.521469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:39.522012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:39.522466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:39.525187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 662/1323] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:39.535747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1694fa-3a1c-4b76-bddd-04161399e488 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:39.536308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1694fa-3a1c-4b76-bddd-04161399e488 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:39.536664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1694fa-3a1c-4b76-bddd-04161399e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:39.616427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0c1694fa-3a1c-4b76-bddd-04161399e488 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:39.617643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c1694fa-3a1c-4b76-bddd-04161399e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:39.687593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:39.736427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 662/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:39.775033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:39.776606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:39.887132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae2d05-1202-45b3-ae45-ed1b588e635f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:39.900527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:39.966132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:39.966762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:39.967051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:39.967325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:39.968962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:40.086489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:40.087885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed37d5e-5a36-4cd4-9687-8f6f6fbec070 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:40.107056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 663/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:42.254025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:42.254636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:42.254636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:42.255997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 663/1326] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:42.261675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f274ab-b87f-4939-b1ea-e723ab173095 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:42.267666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f274ab-b87f-4939-b1ea-e723ab173095 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:42.267666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f274ab-b87f-4939-b1ea-e723ab173095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:42.315279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:42.336220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b7f274ab-b87f-4939-b1ea-e723ab173095 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:42.337438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b7f274ab-b87f-4939-b1ea-e723ab173095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:42.466641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 664/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:37:42.503440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:42.505244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:42.624854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a821a6-2484-4fca-8f42-a0f4d2327cc7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:42.700920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:42.701573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:42.701850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:42.702121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:42.703794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:42.724886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:42.838367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:42.839842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8023e-29fa-4105-9e8c-bd2bb72ea01f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:42.859707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 664/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:45.006173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:45.006753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:45.007087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:45.008583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 665/1329] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:45.014302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c717f1-d685-4063-ba48-52641e329841 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:45.014859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c717f1-d685-4063-ba48-52641e329841 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:45.015110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c717f1-d685-4063-ba48-52641e329841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:45.092045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b2c717f1-d685-4063-ba48-52641e329841 None None] MFA Rules not processed for user `ed5bda805a18497ab72e318c7b4698e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:45.093261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b2c717f1-d685-4063-ba48-52641e329841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:45.162925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:45.200280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 665/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:45.239533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:45.241132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:45.358100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633edfc-e59b-42c4-9401-d48b147861f4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:45.381965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:45.450414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:45.451035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:45.451319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:45.451596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:45.453279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:45.585572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:45.587051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df491ea0-eaac-4cfd-a402-7a8f3dccc8ef service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:45.605086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 666/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:37:48.259892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:48.260461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:48.260461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:48.261829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 666/1332] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:48.267454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b5ad-2f51-4698-be4d-9c70d66da2d6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:48.267900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b5ad-2f51-4698-be4d-9c70d66da2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:48.268017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b5ad-2f51-4698-be4d-9c70d66da2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:48.334954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:48.338293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6a99b5ad-2f51-4698-be4d-9c70d66da2d6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:48.339559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6a99b5ad-2f51-4698-be4d-9c70d66da2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:48.450706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 667/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:48.485135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:48.487753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:48.602175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77418ced-f2d0-4332-92f2-2d875e3a1778 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:48.685014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:48.685748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:48.686032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:48.686308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:48.688048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:48.717972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:48.805570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:48.807024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ec86-260b-46ce-9547-5821cab41340 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:48.825345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 667/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:50.956501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:50.957100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:50.957100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:50.958453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 668/1335] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 10:37:50.968096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5511f8-cf02-4627-90f2-a32a48144191 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:50.968553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5511f8-cf02-4627-90f2-a32a48144191 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:50.968661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5511f8-cf02-4627-90f2-a32a48144191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:51.035742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ab5511f8-cf02-4627-90f2-a32a48144191 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:51.036950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ab5511f8-cf02-4627-90f2-a32a48144191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:51.120981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:51.145020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 668/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:51.185879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:51.187509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:51.295118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9b16b-9175-49c1-94f3-3dd5949e4a96 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:51.327027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:51.375217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:51.376400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:51.376682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:51.376955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:51.378629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:51.493054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:51.494485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab790f-6bc9-4ce6-960c-31c8fb77c1de service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:51.512637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 669/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:54.198391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:54.198973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:54.199069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:54.200380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 669/1338] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:54.206323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6377b-98d0-422f-9e0c-f8ffa1f1bd27 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:54.206744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6377b-98d0-422f-9e0c-f8ffa1f1bd27 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:54.206907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6377b-98d0-422f-9e0c-f8ffa1f1bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:54.270719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-60e6377b-98d0-422f-9e0c-f8ffa1f1bd27 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:54.271919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-60e6377b-98d0-422f-9e0c-f8ffa1f1bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:54.280298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:54.368824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 670/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:54.405816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:54.408443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:54.513378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6fbb4-252a-42d7-b42d-655ea4995631 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:54.587825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:54.588458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:54.588734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:54.589002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:54.590634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:54.628771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:54.703384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:54.705146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1185822-77e6-4b6e-98f7-1ac155fbc294 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:54.723188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 670/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:37:59.140297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:59.140812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:59.140812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:59.146178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 671/1341] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:37:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:37:59.148467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e092-28cd-4d6f-91ec-2006f84980b4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:59.148937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e092-28cd-4d6f-91ec-2006f84980b4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:59.149046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e092-28cd-4d6f-91ec-2006f84980b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:59.232755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bca3e092-28cd-4d6f-91ec-2006f84980b4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:37:59.234240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bca3e092-28cd-4d6f-91ec-2006f84980b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:59.342031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:59.355512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 671/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:37:59.401826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:37:59.403497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:59.514134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd61d5-3e7e-4b3f-a6ec-cd8badd7a677 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:37:59.554127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:37:59.597464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:37:59.598236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:37:59.598593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:37:59.598932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:37:59.600903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:37:59.725996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:37:59.727871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd3796-65bc-449b-a651-2c777fa4f8a3 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:37:59.747699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 672/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:38:03.038712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:03.039273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:03.039273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:03.040529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 672/1344] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:03.045928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07aaeb-647f-40af-ba8d-696c929cfb3a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:03.046382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07aaeb-647f-40af-ba8d-696c929cfb3a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:03.046493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07aaeb-647f-40af-ba8d-696c929cfb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:03.117354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ea07aaeb-647f-40af-ba8d-696c929cfb3a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:03.118616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea07aaeb-647f-40af-ba8d-696c929cfb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:03.137881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:03.237681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 673/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:03.279748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:03.281811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:03.398392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85df90-92b1-4a07-845f-034c0a7a7e90 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:03.487228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:03.487905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:03.488185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:03.488461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:03.490231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:03.540558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:03.615938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:03.617396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f9591-8e4d-4fea-a7ed-a4b23cace910 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:03.637272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 673/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:08.135256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:08.135818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:08.135818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:08.137226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 674/1347] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:08.143511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d812b2-5671-4b66-a659-72d64374e357 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:08.144051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d812b2-5671-4b66-a659-72d64374e357 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:08.144051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d812b2-5671-4b66-a659-72d64374e357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:08.226166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d0d812b2-5671-4b66-a659-72d64374e357 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:08.227425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0d812b2-5671-4b66-a659-72d64374e357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:08.347431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:08.350633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 674/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:08.390870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:08.392521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:08.521821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605cff4-7fae-4e4f-8c73-6d76ad5b836b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:08.588036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:08.627590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:08.628446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:08.628826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:08.629279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:08.631518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:08.767769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:08.769204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfff8b-7b57-4e3d-bdaa-9c9249959e1f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:08.788190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 675/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:13.853061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc711-563c-4596-85e8-5872e99b142f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:13.853573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc711-563c-4596-85e8-5872e99b142f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:13.853573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc711-563c-4596-85e8-5872e99b142f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:13.856035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 675/1350] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:38:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:13.861717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb560-d7c0-4581-9fc6-1725f15b8e6d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:13.862315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb560-d7c0-4581-9fc6-1725f15b8e6d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:13.862315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb560-d7c0-4581-9fc6-1725f15b8e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:13.951376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-936cb560-d7c0-4581-9fc6-1725f15b8e6d None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:13.952784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-936cb560-d7c0-4581-9fc6-1725f15b8e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:13.973818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:14.105077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 676/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:18.414244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188806-d054-4b93-bced-2f5f695e198b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:18.414780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188806-d054-4b93-bced-2f5f695e198b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:18.414891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188806-d054-4b93-bced-2f5f695e198b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:18.416528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 676/1352] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:18.422594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056ebc-3ae5-44fb-aee0-7ff2f8d0c324 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:18.423456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056ebc-3ae5-44fb-aee0-7ff2f8d0c324 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:18.423925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056ebc-3ae5-44fb-aee0-7ff2f8d0c324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:18.506199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d2056ebc-3ae5-44fb-aee0-7ff2f8d0c324 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:18.507834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2056ebc-3ae5-44fb-aee0-7ff2f8d0c324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:18.588469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:18.631172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 677/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:20.026615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96177b5e-6879-4cc3-9fb8-ba0b899d67a3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:20.026615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96177b5e-6879-4cc3-9fb8-ba0b899d67a3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:20.026615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96177b5e-6879-4cc3-9fb8-ba0b899d67a3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:20.027314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 677/1354] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:20.032378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa19f-6e8d-4eba-8cdc-43d6b5e5e541 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:20.032888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa19f-6e8d-4eba-8cdc-43d6b5e5e541 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:20.032985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa19f-6e8d-4eba-8cdc-43d6b5e5e541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:20.117080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1b7aa19f-6e8d-4eba-8cdc-43d6b5e5e541 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:20.118302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1b7aa19f-6e8d-4eba-8cdc-43d6b5e5e541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:20.243809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 678/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:38:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:24.022400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196791ae-8759-4b0a-a04a-9ea1d66e6bc8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:24.022964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196791ae-8759-4b0a-a04a-9ea1d66e6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:24.022964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196791ae-8759-4b0a-a04a-9ea1d66e6bc8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:24.024167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 678/1356] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:24.031886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a813a2-d40d-43bc-95a1-c66ff7b0013d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:24.032402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a813a2-d40d-43bc-95a1-c66ff7b0013d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:24.032935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a813a2-d40d-43bc-95a1-c66ff7b0013d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:24.078583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:24.116281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-67a813a2-d40d-43bc-95a1-c66ff7b0013d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:24.117484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-67a813a2-d40d-43bc-95a1-c66ff7b0013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:24.252751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 679/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:24.328987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:24.330114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:24.330949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:24.333842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 679/1358] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 21 10:38:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:24.341847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e2ff-f942-4129-b91b-cdbb9cb6f17c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:24.342498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e2ff-f942-4129-b91b-cdbb9cb6f17c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:24.342998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e2ff-f942-4129-b91b-cdbb9cb6f17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:24.429531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9936e2ff-f942-4129-b91b-cdbb9cb6f17c None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:24.430989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9936e2ff-f942-4129-b91b-cdbb9cb6f17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:24.488838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:24.561083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 680/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 21 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:24.572198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:24.574039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:24.740230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454e028-c132-4c6f-a658-0fba932018a0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:24.823268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:24.872790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:24.873604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:24.873982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:24.874350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:24.876440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:24.976089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:24.977840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1cc5-2d2b-45c3-a7b6-5376adaa4082 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:24.997806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 680/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:27.424959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:27.425497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:27.425497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:27.428004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 681/1361] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:27.431799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd9e0-3fec-4a3b-8606-b341704498cb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:27.432193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd9e0-3fec-4a3b-8606-b341704498cb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:27.432400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd9e0-3fec-4a3b-8606-b341704498cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:27.503604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-895fd9e0-3fec-4a3b-8606-b341704498cb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:27.505070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-895fd9e0-3fec-4a3b-8606-b341704498cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:27.548729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:27.618836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 681/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:27.665127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:27.667124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:27.885506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abe523-c2a9-43db-9ae2-36c8789c2e37 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:27.899580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:28.058685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:28.059224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:28.059509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:28.059784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:28.061448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:28.180020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:28.181826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1d9b6-5eb2-4e5d-858a-8323ab1e4d27 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:28.207161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 682/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:30.720258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dbad-82b9-4418-87b9-dc480b40fa4f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:30.720769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dbad-82b9-4418-87b9-dc480b40fa4f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:30.720769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dbad-82b9-4418-87b9-dc480b40fa4f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:30.722262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 682/1364] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:30.727350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0de59-c6a2-4293-ae8f-75ab2e9dd137 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:30.727758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0de59-c6a2-4293-ae8f-75ab2e9dd137 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:30.727978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0de59-c6a2-4293-ae8f-75ab2e9dd137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:30.743665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:30.798359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-00f0de59-c6a2-4293-ae8f-75ab2e9dd137 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:30.799902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-00f0de59-c6a2-4293-ae8f-75ab2e9dd137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:30.910592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 683/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:30.956016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:30.956299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:30.956768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:30.958276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 683/1366] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 21 10:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:30.963633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227eb-5c0e-4e34-8a86-3d1423c53e97 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:30.969154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227eb-5c0e-4e34-8a86-3d1423c53e97 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:30.969154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227eb-5c0e-4e34-8a86-3d1423c53e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:31.045690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d9e227eb-5c0e-4e34-8a86-3d1423c53e97 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:31.047053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d9e227eb-5c0e-4e34-8a86-3d1423c53e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:31.086484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:31.178748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 684/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 21 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:31.186891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:31.192494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:31.349391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4b724-d121-4da4-98b5-966eb28b5812 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:31.370874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:31.475295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:31.475974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:31.476249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:31.476517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:31.478146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:31.576926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:31.579054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a92e-4bb6-403a-8547-dbaac85701c7 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:31.600507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 684/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:32.524492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1295b-0eeb-4f68-ac49-ad35e9eaafaa None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:32.525007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1295b-0eeb-4f68-ac49-ad35e9eaafaa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:32.525007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1295b-0eeb-4f68-ac49-ad35e9eaafaa None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:32.526306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 685/1369] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:32.533293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d93bf-a52a-4eea-8ec9-149dd1a221a8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:32.533995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d93bf-a52a-4eea-8ec9-149dd1a221a8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:32.534373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d93bf-a52a-4eea-8ec9-149dd1a221a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:32.580558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:32.618134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-964d93bf-a52a-4eea-8ec9-149dd1a221a8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:32.619619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-964d93bf-a52a-4eea-8ec9-149dd1a221a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:32.734602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 685/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:33.092956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:33.092956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:33.092956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:33.093965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 686/1371] 10.222.0.48 () {54 vars in 964 bytes} [Thu Jul 21 10:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:33.098921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3145c8-c66b-4f2b-8067-e419b15027ad None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:33.099210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3145c8-c66b-4f2b-8067-e419b15027ad None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:33.099485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3145c8-c66b-4f2b-8067-e419b15027ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:33.186085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4e3145c8-c66b-4f2b-8067-e419b15027ad None None] MFA Rules not processed for user `1ec420ff94214b3ab6a041395c0a491c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:33.190799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4e3145c8-c66b-4f2b-8067-e419b15027ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:33.240707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:33.273289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 686/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Jul 21 10:38:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:33.281471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:33.283153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:33.397243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f2bb4-f83d-471f-a9a1-dfd2c77e0148 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:33.443600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:33.504278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:33.505423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:33.505914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:33.506408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:33.509618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:33.637887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:33.639532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac67c1-dfc2-4312-abdd-7439dc87b76d service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:33.660057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 687/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:33.928928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:33.929653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:33.929653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:33.934725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 687/1374] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:33.936082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438871-1920-4b35-8954-0e77a43a1047 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:33.936566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438871-1920-4b35-8954-0e77a43a1047 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:33.936746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438871-1920-4b35-8954-0e77a43a1047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:34.011774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a5438871-1920-4b35-8954-0e77a43a1047 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:34.013054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a5438871-1920-4b35-8954-0e77a43a1047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:34.036783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:34.120579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 688/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:34.166434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:34.167983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:34.338433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c6271-90dd-4af0-b28b-ae525fec4525 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:34.392165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:34.456755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:34.457352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:34.457636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:34.457913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:34.459644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:34.553637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:34.555281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae93482-bca1-49b6-bce3-7cfce1f30768 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:34.574518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 688/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:36.814254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:36.814864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:36.814864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:36.816351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 689/1377] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:36.821662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579a30-6115-4897-bd92-e157c8063e41 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:36.822093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579a30-6115-4897-bd92-e157c8063e41 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:36.822222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579a30-6115-4897-bd92-e157c8063e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:36.894498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ef579a30-6115-4897-bd92-e157c8063e41 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:36.895502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ef579a30-6115-4897-bd92-e157c8063e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:36.903696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:37.019737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 689/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:37.072414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:37.073912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:37.249361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fc4ee-b379-4a54-b6a0-b8723cd708dc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:37.257224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:37.415936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:37.415936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:37.415936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:37.415936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:37.417329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:37.532249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:37.534041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db6a0b-ff77-48c2-aa68-99a7c8759fc9 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:37.553385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 690/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:39.873989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:39.874846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:39.874846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:39.878931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 690/1380] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:39.881370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b996-03ab-4478-a165-dbfdc6f5830b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:39.881851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b996-03ab-4478-a165-dbfdc6f5830b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:39.881984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b996-03ab-4478-a165-dbfdc6f5830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:39.893174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:39.958131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c6e4b996-03ab-4478-a165-dbfdc6f5830b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:39.959638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c6e4b996-03ab-4478-a165-dbfdc6f5830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:40.071379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 691/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:40.116342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:40.118525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:40.289487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:40.301729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42380013-b4ad-4087-98ce-0b11a45775fc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:40.448505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:40.449196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:40.449482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:40.449757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:40.451475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:40.553962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:40.555931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7019-0755-4389-a36e-f471eb2f559f service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:40.568086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:40.577962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 691/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:42.898247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:42.898856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:42.898856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:42.900294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 692/1383] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:42.906164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b0f7-db1a-4e40-b542-ca2ad427cec1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:42.906903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b0f7-db1a-4e40-b542-ca2ad427cec1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:42.906903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b0f7-db1a-4e40-b542-ca2ad427cec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:42.995666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-79b1b0f7-db1a-4e40-b542-ca2ad427cec1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:42.996896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-79b1b0f7-db1a-4e40-b542-ca2ad427cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:43.144628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 692/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:43.188960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:43.190375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:43.314689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:43.383645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3ec4f-fe03-454d-bf8c-1b2a3acb5069 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:43.519141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:43.519853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:43.520393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:43.520747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:43.522975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:43.596329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:43.639476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:43.641144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453e8a1-e069-4ab5-8683-2eac7461c184 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:43.674963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 693/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:45.834628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:45.835195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:45.835195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:45.836576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 693/1386] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:45.845048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993564-229d-46bc-b497-e148b7c9b06a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:45.845835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993564-229d-46bc-b497-e148b7c9b06a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:45.845835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993564-229d-46bc-b497-e148b7c9b06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:45.929689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-44993564-229d-46bc-b497-e148b7c9b06a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:45.931286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44993564-229d-46bc-b497-e148b7c9b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:46.045419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 694/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:46.102237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:46.103814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:46.210725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:46.323894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a058-ab60-4ea8-b21c-4b11a0eb3723 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:46.466045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:46.466656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:46.466937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:46.467211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:46.469604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:46.522189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:46.594160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:46.596541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba7270-3ab0-40d8-a5b0-8071133fec83 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:46.620904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 694/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:48.917328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689456-258b-4921-8248-7fabed680575 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:48.917894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689456-258b-4921-8248-7fabed680575 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:48.917894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689456-258b-4921-8248-7fabed680575 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:48.919565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 695/1389] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:48.925779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaefd-3039-429c-9246-8739068088e9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:48.926588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaefd-3039-429c-9246-8739068088e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:48.926588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaefd-3039-429c-9246-8739068088e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:48.996878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fc0eaefd-3039-429c-9246-8739068088e9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:48.998247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fc0eaefd-3039-429c-9246-8739068088e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:49.101833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 695/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:49.146190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5689456-258b-4921-8248-7fabed680575 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:49.147846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5689456-258b-4921-8248-7fabed680575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:49.182647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:49.300703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5689456-258b-4921-8248-7fabed680575 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:49.402202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:49.414578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:49.417273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:49.417273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:49.417273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:49.417576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:49.509835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:49.511377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66f18c-9ea5-42a1-8153-0ab53ae34396 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:49.533635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 696/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:49.949461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-3ffb-4f8b-ac3e-dc96910b7806 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:49.950017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-3ffb-4f8b-ac3e-dc96910b7806 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:49.950194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-3ffb-4f8b-ac3e-dc96910b7806 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:49.952733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 696/1392] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:49.960497 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b197-92d7-4325-b051-91fe0c16f1b4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:49.961155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b197-92d7-4325-b051-91fe0c16f1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:49.961261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b197-92d7-4325-b051-91fe0c16f1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:50.034790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a5c0b197-92d7-4325-b051-91fe0c16f1b4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:50.036025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a5c0b197-92d7-4325-b051-91fe0c16f1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:50.102098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:50.147854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 697/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:50.205505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:50.205792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:50.206071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:50.209761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 697/1394] 10.222.0.48 () {54 vars in 966 bytes} [Thu Jul 21 10:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:50.213025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae9196-0318-4a55-8f54-019c34c36071 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:50.213311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae9196-0318-4a55-8f54-019c34c36071 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:50.213586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae9196-0318-4a55-8f54-019c34c36071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:50.290817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-88ae9196-0318-4a55-8f54-019c34c36071 None None] MFA Rules not processed for user `ebd9b2da29ee4612afb617183145553b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:50.292031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-88ae9196-0318-4a55-8f54-019c34c36071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:50.370365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 698/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Thu Jul 21 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:50.378978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:50.380502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:50.395541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:50.495247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed0ef1-4e7e-4faf-b9e3-d0c66b42f92b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:50.569434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:50.570101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:50.570393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:50.570689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:50.572436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:50.645286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:50.687217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:50.688641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74927d67-1b68-4d14-a193-7683520c216e service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:50.707952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 698/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:51.573946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca0a41-6823-4498-84d0-e55ecfbf4ebc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:51.574458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca0a41-6823-4498-84d0-e55ecfbf4ebc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:51.574580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca0a41-6823-4498-84d0-e55ecfbf4ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:51.649561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-21ca0a41-6823-4498-84d0-e55ecfbf4ebc None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:51.650833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-21ca0a41-6823-4498-84d0-e55ecfbf4ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:51.661280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:51.747136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 699/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:51.794141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba4ba-fd82-4a54-b61e-7bf06a2787fb None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:51.794673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba4ba-fd82-4a54-b61e-7bf06a2787fb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:51.794780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba4ba-fd82-4a54-b61e-7bf06a2787fb None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:51.796686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 699/1398] 10.222.0.48 () {54 vars in 943 bytes} [Thu Jul 21 10:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:51.802631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02879-983d-449f-bfbf-157cd57fdfa8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:51.802925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02879-983d-449f-bfbf-157cd57fdfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:51.803143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:51.803248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02879-983d-449f-bfbf-157cd57fdfa8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:51.803536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:51.803869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:51.804563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 700/1399] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:51.809719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:51.809941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:51.810218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:51.880564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:51.881944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:51.888476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] MFA Rules not processed for user `7b4c6a5adbda4403883661e072aba906`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:51.889731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:51.988355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 701/1400] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:52.009440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 700/1401] 10.222.0.48 () {58 vars in 1040 bytes} [Thu Jul 21 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:52.025157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:52.026828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:52.030013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:52.035183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:52.036428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:52.039593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:52.142929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557dc56-5066-4210-be7a-0d6127062958 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:52.150490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658ff41-7635-423a-9715-a9c9145e8a17 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:52.268607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:52.269259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:52.269535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:52.269804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:52.271608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:52.278450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:52.279129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:52.279407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:52.279682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:52.282712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:52.296034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:52.300044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:52.372324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:52.374262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ab803-2281-4210-b04b-873fd4b3da8a service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:52.383314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 702/1402] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:38:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:38:52.385744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:52.387508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152711c-7326-4a8e-964b-60d1041650fc service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:52.411806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 701/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:38:54.680706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:54.681328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:54.681328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:54.683579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 703/1404] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:54.689907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba046-88f1-4e6a-9d9c-7ccdf6bb19f4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:54.690440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba046-88f1-4e6a-9d9c-7ccdf6bb19f4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:54.690777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba046-88f1-4e6a-9d9c-7ccdf6bb19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:54.783933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a7cba046-88f1-4e6a-9d9c-7ccdf6bb19f4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:54.785279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a7cba046-88f1-4e6a-9d9c-7ccdf6bb19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:54.905337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:54.920470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 702/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:38:54.968759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:54.970356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:55.293045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d51b9-af1e-4ed5-bff6-3665a3231bd4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:55.313736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:55.424130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:55.424698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:55.424979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:55.425277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:55.426936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:55.541079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:55.542712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3ef18-8f76-4876-884a-0d0c864d63bc service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:55.566496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 704/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:38:57.979543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:57.980123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:57.980123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:57.981473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 703/1407] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:38:57.987428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85704b-7149-4cf6-8d9e-77e977918148 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:57.987921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85704b-7149-4cf6-8d9e-77e977918148 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:57.988026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85704b-7149-4cf6-8d9e-77e977918148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:58.021224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:58.071872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ff85704b-7149-4cf6-8d9e-77e977918148 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:38:58.075297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ff85704b-7149-4cf6-8d9e-77e977918148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:58.204658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 705/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:38:58.252338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:38:58.254292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:58.437462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ed405-40df-46e4-8f81-528c68072f9f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:38:58.533280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:38:58.591828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:38:58.594873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:38:58.594873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:38:58.594873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:38:58.595371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:38:58.708457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:38:58.710449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d568a25-8c88-4a95-8cf8-844d8550fcbf service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:38:58.736422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 704/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:08.202456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=56 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:08.733029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=66 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:15.962955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791e0d3-0bb3-4326-a01f-3f90dc90fc52 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:15.963513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791e0d3-0bb3-4326-a01f-3f90dc90fc52 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:15.963513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791e0d3-0bb3-4326-a01f-3f90dc90fc52 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:15.964866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 706/1410] 10.222.0.48 () {54 vars in 918 bytes} [Thu Jul 21 10:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:15.970118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a10f50-aead-4b2c-bffe-f224cbd615bb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:15.970574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a10f50-aead-4b2c-bffe-f224cbd615bb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:15.970684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a10f50-aead-4b2c-bffe-f224cbd615bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:16.044777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-60a10f50-aead-4b2c-bffe-f224cbd615bb None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:16.045996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-60a10f50-aead-4b2c-bffe-f224cbd615bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:16.151832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 705/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 21 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:16.166588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b87c7-754e-4f03-8fb4-e63fe0d2be3c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:16.166877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b87c7-754e-4f03-8fb4-e63fe0d2be3c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:16.167150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b87c7-754e-4f03-8fb4-e63fe0d2be3c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:16.168493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 707/1412] 10.222.0.48 () {54 vars in 918 bytes} [Thu Jul 21 10:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:16.174136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1503-b660-472a-ba58-3dc38d50416e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:16.174464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1503-b660-472a-ba58-3dc38d50416e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:16.174777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1503-b660-472a-ba58-3dc38d50416e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:16.246180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:16.249720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8e2b1503-b660-472a-ba58-3dc38d50416e None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:16.251001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8e2b1503-b660-472a-ba58-3dc38d50416e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:16.367531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 706/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 21 10:39:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:18.908621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:18.909193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:18.909193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:18.910591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:18.915837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37725-b072-4aa6-bcec-1bab592eaeeb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:18.916293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37725-b072-4aa6-bcec-1bab592eaeeb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:18.916399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37725-b072-4aa6-bcec-1bab592eaeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:18.983178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cec37725-b072-4aa6-bcec-1bab592eaeeb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:18.984409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cec37725-b072-4aa6-bcec-1bab592eaeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:19.040871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:19.085033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:19.129271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:19.130801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:19.330339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaca06-2d6a-4a5e-9001-1f607becf577 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:19.405510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:19.447592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:19.448152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:19.448432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:19.448702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:19.450347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:19.540905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:19.542481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a57e2-024c-48c5-a13e-55f13767eabc service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:19.560953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 709/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:20.480784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:20.481318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:20.481318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:20.483087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:20.488157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394899-0ec9-423a-841e-b3617ce96fae None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:20.488583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394899-0ec9-423a-841e-b3617ce96fae None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:20.488711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc394899-0ec9-423a-841e-b3617ce96fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:20.563228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cc394899-0ec9-423a-841e-b3617ce96fae None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:20.564447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cc394899-0ec9-423a-841e-b3617ce96fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:20.591414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:20.664764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:20.817726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:20.819176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:20.971328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba78d3-cb13-426f-bd8a-4b3701a4af88 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:21.012001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:21.116096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:21.116985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:21.117426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:21.117864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:21.120623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:21.229363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:21.232704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12925b82-857e-43f0-bbd9-aaf37aa3dbc8 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:21.257566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 709/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:22.162880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:22.163439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:22.163439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:22.165193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:22.171004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249cbc5-f1fc-4604-8e52-2d2f64ef6e02 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:22.171004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249cbc5-f1fc-4604-8e52-2d2f64ef6e02 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:22.171559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249cbc5-f1fc-4604-8e52-2d2f64ef6e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:22.230871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:22.259293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6249cbc5-f1fc-4604-8e52-2d2f64ef6e02 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:22.260522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6249cbc5-f1fc-4604-8e52-2d2f64ef6e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:22.361075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:22.515041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:22.516538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:22.666231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137496a-aa10-43b2-86ac-5292470a611e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:22.679191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:22.784210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:22.784948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:22.785224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:22.785493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:22.787306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:22.885899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:22.887539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f83ff8-cdf4-413a-89e2-ff0d91803287 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:22.907889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 712/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:23.804455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:23.804977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:23.804977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:23.806303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:23.811821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d809e-6f4e-471f-a054-96d035110d9a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:23.812556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d809e-6f4e-471f-a054-96d035110d9a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:23.812556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d809e-6f4e-471f-a054-96d035110d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:23.842340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:23.925517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e68d809e-6f4e-471f-a054-96d035110d9a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:23.927414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e68d809e-6f4e-471f-a054-96d035110d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:24.042047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:39:24.206199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:24.208604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:24.323108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:24.378571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a52bc-ac24-49e9-b8f9-531512bb528d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:24.492233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:24.492799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:24.493080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:24.493355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:24.495065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:24.571657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:24.586082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:24.587724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdb50-c75d-4e7e-a01e-e6596b9d5d86 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:24.608443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 712/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:25.509534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:25.510079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:25.510079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:25.511395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:25.517741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ceed-a9d7-4ed7-891a-d3aa38527eaf None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:25.518233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ceed-a9d7-4ed7-891a-d3aa38527eaf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:25.518340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ceed-a9d7-4ed7-891a-d3aa38527eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:25.599492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2dd9ceed-a9d7-4ed7-891a-d3aa38527eaf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:25.600734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2dd9ceed-a9d7-4ed7-891a-d3aa38527eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:25.704087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:25.865948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:25.867416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:25.953545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:26.016308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9304c70-18a6-4e92-b260-41c018f9759e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:26.127448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:26.128003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:26.128278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:26.128545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:26.130167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:26.178927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:26.221575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:26.223254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d715c-2dc3-47e6-8cf0-631191bd5917 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:26.242120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 715/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:27.132244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:27.132900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:27.132900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:27.134171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:27.139304 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097302a1-7e06-4f38-9e23-714cbb66f0bb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:27.139774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097302a1-7e06-4f38-9e23-714cbb66f0bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:27.139886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097302a1-7e06-4f38-9e23-714cbb66f0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:27.214635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-097302a1-7e06-4f38-9e23-714cbb66f0bb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:27.226504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-097302a1-7e06-4f38-9e23-714cbb66f0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:27.316518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:27.468358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:27.469998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:27.518725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:27.613832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a9e34-21e3-4ebe-972d-347c2c027480 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:27.725538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:27.726113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:27.726398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:27.726703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:27.728473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:27.735848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:27.821506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:27.823149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf036dd4-7c11-4ba8-927d-6e4a23a4af0e service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:27.841490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 715/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:28.714099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:28.714776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:28.714776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:28.716075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:28.721870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42e1c-5845-4ca1-aefb-8b12f33fe5dc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:28.722381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42e1c-5845-4ca1-aefb-8b12f33fe5dc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:28.722496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42e1c-5845-4ca1-aefb-8b12f33fe5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:28.804219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-77b42e1c-5845-4ca1-aefb-8b12f33fe5dc None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:28.805427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-77b42e1c-5845-4ca1-aefb-8b12f33fe5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:28.875152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:28.915318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:29.069393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:29.070899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:29.088589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:29.225904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc3988-c425-4b3f-9d8c-b0c15e334185 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:29.314873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:29.340558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:29.341149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:29.341937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:29.342287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:29.343974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:29.435446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:29.437047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876ecf-3ebd-41e6-adc1-5db4f4377691 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:29.455260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 718/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:30.367624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:30.368096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:30.368223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:30.369665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:30.376465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf486e-6fa6-4e33-9b32-163bfba6e6c7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:30.376968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf486e-6fa6-4e33-9b32-163bfba6e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:30.377074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf486e-6fa6-4e33-9b32-163bfba6e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:30.454848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b5cf486e-6fa6-4e33-9b32-163bfba6e6c7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:30.455951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b5cf486e-6fa6-4e33-9b32-163bfba6e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:30.499548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:30.555954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:30.712234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:30.713739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:30.863779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef32ecd-d020-4149-8b23-93320a60a450 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:30.926127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:30.987866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:30.988426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:30.988705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:30.988979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:30.990717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:31.086227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:31.087880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216ceec-1fd1-4f22-a054-209239026a09 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:31.105723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 718/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:31.977082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:31.977652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:31.977652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:31.979026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:31.984890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7786-e3c0-4e09-a886-5b8dc0613f7a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:31.985335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7786-e3c0-4e09-a886-5b8dc0613f7a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:31.985474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7786-e3c0-4e09-a886-5b8dc0613f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:32.063721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1e6a7786-e3c0-4e09-a886-5b8dc0613f7a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:32.064929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1e6a7786-e3c0-4e09-a886-5b8dc0613f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:32.067058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:32.171888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:39:32.324820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:32.326345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:32.479187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105b9ad-b45a-4c9a-8366-392f612293b1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:32.507606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:32.597252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:32.597810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:32.598090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:32.598364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:32.600082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:32.692266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:32.693862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f9f7e-f095-41cd-9ea8-76eaa31a5536 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:32.711966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 721/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:34.706761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:34.707342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:34.707342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:34.708827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:34.714645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab1d5-0936-4d4f-a8bf-dfc50a4e4bc1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:34.715154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab1d5-0936-4d4f-a8bf-dfc50a4e4bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:34.715301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab1d5-0936-4d4f-a8bf-dfc50a4e4bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:34.756535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:34.783447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b92ab1d5-0936-4d4f-a8bf-dfc50a4e4bc1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:34.784665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b92ab1d5-0936-4d4f-a8bf-dfc50a4e4bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:34.887148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:34.928160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:34.929724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:35.038924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999c187-5fb2-4256-9368-c469364e9047 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:35.073780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:35.116494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:35.117144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:35.117429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:35.117709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:35.119393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:35.230485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:35.232260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708bc55-c999-4f4a-add5-c3be0d48e609 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:35.251995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 721/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:37.483899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:37.484474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:37.484474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:37.485728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:37.491149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3563f29-f008-41fc-b74d-51f3eee93f63 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:37.491606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3563f29-f008-41fc-b74d-51f3eee93f63 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:37.491720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3563f29-f008-41fc-b74d-51f3eee93f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:37.557821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f3563f29-f008-41fc-b74d-51f3eee93f63 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:37.559080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f3563f29-f008-41fc-b74d-51f3eee93f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:37.573772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:37.661779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:39:37.701998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:37.703560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:37.823316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398977b-8ffa-46af-bd0f-a4948604ff9e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:37.836174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:37.898875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:37.899518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:37.899795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:37.900067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:37.901691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:38.023484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:38.025006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb4c18-8e09-41a3-b814-8751abc06981 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:38.045875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 724/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:40.506407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:40.506982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:40.506982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:40.508277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:40.514678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f399a-e13d-406e-99f1-ec9bc9189210 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:40.515098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f399a-e13d-406e-99f1-ec9bc9189210 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:40.515235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f399a-e13d-406e-99f1-ec9bc9189210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:40.563255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:40.581130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-974f399a-e13d-406e-99f1-ec9bc9189210 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:40.582325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-974f399a-e13d-406e-99f1-ec9bc9189210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:40.683100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:40.722661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:40.724197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:40.830786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9224b3-c1da-4225-8233-ba9cd3f246ee None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:40.870252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:40.904779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:40.905436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:40.905714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:40.905982 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:40.908584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:41.023156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:41.024594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127366b-807d-4414-a353-85476da2d878 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:41.045073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 724/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:43.253558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:43.254094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:43.254094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:43.255597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:43.261275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b1d8-846e-431a-8a27-136232b32633 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:43.261699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b1d8-846e-431a-8a27-136232b32633 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:43.261964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b1d8-846e-431a-8a27-136232b32633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:43.330650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3278b1d8-846e-431a-8a27-136232b32633 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:43.331850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3278b1d8-846e-431a-8a27-136232b32633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:43.353082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:43.432238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:43.472627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:43.474147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:43.580672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca62c-fe97-4fd5-a433-98a2c7543f95 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:43.601999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:43.661352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:43.662303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:43.662621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:43.662897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:43.664773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:43.788834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:43.790191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bad72-fd12-401f-b289-8738e5d12db8 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:43.809107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 727/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:46.003521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:46.004087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:46.004087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:46.005428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:46.010925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d25c8-7abc-4d1e-87be-37849db9bb1f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:46.011588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d25c8-7abc-4d1e-87be-37849db9bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:46.011588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d25c8-7abc-4d1e-87be-37849db9bb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:46.078848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:46.081960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-af3d25c8-7abc-4d1e-87be-37849db9bb1f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:46.083219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-af3d25c8-7abc-4d1e-87be-37849db9bb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:46.183990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:46.223841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:46.225386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:46.335282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9740d6-c560-40d6-8001-8506e0674e73 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:46.388522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:46.416929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:46.417601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:46.417877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:46.418145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:46.420524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:46.531575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:46.533001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37003bb-a165-4d29-a4be-3bcb8e927ba7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:46.552264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 727/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:48.697997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:48.698607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:48.698607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:48.699985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:48.706039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9a72f-3730-487a-988d-dd3b213f9aea None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:48.706596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9a72f-3730-487a-988d-dd3b213f9aea None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:48.706704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9a72f-3730-487a-988d-dd3b213f9aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:48.777095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-14b9a72f-3730-487a-988d-dd3b213f9aea None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:48.778428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-14b9a72f-3730-487a-988d-dd3b213f9aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:48.808910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:48.881880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:48.922198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:48.923771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:49.035459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaee6f-65c6-40e7-9e28-a9f23e04a069 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:49.066022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:49.115726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:49.116380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:49.116665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:49.117072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:49.118752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:49.237146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:49.238851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129207-9eb4-454d-911c-0d5ef4c4bf4c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:49.260154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 730/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:51.393894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:51.394481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:51.394481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:51.395927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:51.401375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cc84-3973-4f92-b789-8e9a39d1cae7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:51.401827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cc84-3973-4f92-b789-8e9a39d1cae7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:51.401934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cc84-3973-4f92-b789-8e9a39d1cae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:51.473941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-83e9cc84-3973-4f92-b789-8e9a39d1cae7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:51.475106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-83e9cc84-3973-4f92-b789-8e9a39d1cae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:51.483979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:51.572893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:51.613091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:51.614666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:51.724031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37baf-e2e3-4aa7-abdd-4c42dc1f4593 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:51.782375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:51.803024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:51.803787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:51.804067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:51.804339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:51.805988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:51.921063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:51.922500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aec2aa-d6b3-4f9c-b30b-a90a455d87bc service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:51.941799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 730/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:54.131369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:54.131949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:54.131949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:54.133265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:54.139653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e152-1eed-4ff8-8412-94e5eecb8816 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:54.140177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e152-1eed-4ff8-8412-94e5eecb8816 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:54.140318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e152-1eed-4ff8-8412-94e5eecb8816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:54.214021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b7d6e152-1eed-4ff8-8412-94e5eecb8816 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:54.215264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b7d6e152-1eed-4ff8-8412-94e5eecb8816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:54.259194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:54.322777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:54.362906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:54.364480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:54.471626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef92f2-02a9-423d-a20b-6dd4943c8014 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:54.510580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:54.547229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:54.548211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:54.548560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:54.549131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:54.550815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:54.661808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:54.663276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422f9dd-098d-4d6f-8ec1-a0fc69a3e9fb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:54.682499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 733/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:56.786210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:56.786812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:56.786812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:56.788168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:56.793642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892ce7e-5927-4b27-86d0-737a7659c480 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:56.794261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892ce7e-5927-4b27-86d0-737a7659c480 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:56.794565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892ce7e-5927-4b27-86d0-737a7659c480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:56.861692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b892ce7e-5927-4b27-86d0-737a7659c480 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:56.862850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b892ce7e-5927-4b27-86d0-737a7659c480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:56.881314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:56.963785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:57.002427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:57.004067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:57.112154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4e499-a71b-46fe-b40a-b9b03d0df247 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:57.182888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:57.190577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:57.191232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:57.193518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:57.193518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:57.193518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:57.307630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:57.309077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f95a-6cc3-4c30-a4dd-46eb20885611 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:57.328925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 733/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:39:59.417255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:59.417833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:59.417833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:59.419319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:39:59.424595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96588a-a764-420d-97b1-888aa052d10b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:59.425058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96588a-a764-420d-97b1-888aa052d10b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:59.425168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96588a-a764-420d-97b1-888aa052d10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:59.490086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-db96588a-a764-420d-97b1-888aa052d10b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:39:59.491388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-db96588a-a764-420d-97b1-888aa052d10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:59.541452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:59.593081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:39:59.630910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:39:59.632419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:59.740986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616e4a-0416-451b-983b-9c588cb8182c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:39:59.790914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:39:59.817585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:39:59.818246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:39:59.818524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:39:59.818933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:39:59.820579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:39:59.930427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:39:59.931892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff169554-bd32-4e47-b7da-3510d6bb214f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:39:59.951160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 736/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:09.590744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=22 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:09.948125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=66 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:21.064656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead6b0f-6acf-4d0c-96a9-835b4fc72077 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:21.065262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead6b0f-6acf-4d0c-96a9-835b4fc72077 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:21.065262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead6b0f-6acf-4d0c-96a9-835b4fc72077 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:21.066625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 735/1471] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:21.072013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35631c-0de5-4b2f-b365-dd82fb26d9a8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:21.072451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35631c-0de5-4b2f-b365-dd82fb26d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:21.072554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35631c-0de5-4b2f-b365-dd82fb26d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:21.136926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0d35631c-0de5-4b2f-b365-dd82fb26d9a8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:21.138089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0d35631c-0de5-4b2f-b365-dd82fb26d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:21.235444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 737/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:21.324703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:21.325077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:21.325569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:21.327002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 736/1473] 10.222.0.48 () {54 vars in 966 bytes} [Thu Jul 21 10:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:21.331930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340230f0-fc89-4a72-a1fd-2efe0b026750 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:21.332220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340230f0-fc89-4a72-a1fd-2efe0b026750 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:21.332495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340230f0-fc89-4a72-a1fd-2efe0b026750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:21.408854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-340230f0-fc89-4a72-a1fd-2efe0b026750 None None] MFA Rules not processed for user `ebd9b2da29ee4612afb617183145553b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:21.410067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-340230f0-fc89-4a72-a1fd-2efe0b026750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:21.430332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:21.487050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 738/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Thu Jul 21 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:21.494250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:21.495860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:21.600051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1465cf0-883f-48fd-b466-d93c71358e10 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:21.673509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:21.674158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:21.674440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:21.674744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:21.676405 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:21.730267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:21.786278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:21.787753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb623e8-4611-4fb2-bb4d-ecfe3b0689e9 service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:21.806027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 737/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:23.963845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:23.964418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:23.964418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:23.966410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 739/1476] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:23.973068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8103a-ef20-4390-bf4b-616d667dbf03 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:23.973541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8103a-ef20-4390-bf4b-616d667dbf03 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:23.973701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8103a-ef20-4390-bf4b-616d667dbf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:24.047043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3ad8103a-ef20-4390-bf4b-616d667dbf03 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:24.048302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3ad8103a-ef20-4390-bf4b-616d667dbf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:24.147888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 738/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:24.173088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:24.174730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:24.314315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797a865-5a5e-476b-8854-c08f082a9ff4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:24.363658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:24.410105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:24.410830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:24.411114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:24.411387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:24.412110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:24.413892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:40:24.413892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:40:24.413892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:40:24.415066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500290e8-0f9b-4cea-ac6a-9756dbccbb3f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:24.435274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 740/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 21 10:40:24 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:40:26.624437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:26.625189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:26.625189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:26.626432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 739/1479] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:26.631749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fed-92d8-4ed6-a0e3-105510048b56 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:26.632306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fed-92d8-4ed6-a0e3-105510048b56 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:26.632306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fed-92d8-4ed6-a0e3-105510048b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:26.696754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-98674fed-92d8-4ed6-a0e3-105510048b56 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:26.697966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98674fed-92d8-4ed6-a0e3-105510048b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:26.794654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 741/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:26.843624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:26.845176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:26.854988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:26.993248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd4dbc-a1fb-417f-aad3-7d79523e350c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:27.080473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:27.124470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:27.125482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:27.125996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:27.126490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:27.129776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:27.235125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:27.236724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfecb6-8a35-4cda-994d-2f5c7a50adf6 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:27.255169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 740/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:29.545873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:29.546444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:29.546444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:29.547748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 742/1482] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:29.553140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f7f1-d29c-4adc-b7d1-374f1509a266 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:29.553550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f7f1-d29c-4adc-b7d1-374f1509a266 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:29.553690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419f7f1-d29c-4adc-b7d1-374f1509a266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:29.618443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e419f7f1-d29c-4adc-b7d1-374f1509a266 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:29.619669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e419f7f1-d29c-4adc-b7d1-374f1509a266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:29.666961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:29.718796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 741/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:40:29.755953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:29.757732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:29.783067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:29.871752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdcf7d-4bb6-4c30-984a-02d2dfa27408 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:29.947492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:29.948130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:29.948407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:29.948677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:29.951172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:30.030636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:30.068546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:30.069975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fa396-4289-420c-ae6d-1a4daf2fddcf service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:30.091292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 743/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:32.228902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:32.229470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:32.229470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:32.232301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 742/1485] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:32.237596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78d44c-9dfe-4812-977b-57fd7136ae97 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:32.238055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78d44c-9dfe-4812-977b-57fd7136ae97 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:32.238158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78d44c-9dfe-4812-977b-57fd7136ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:32.309219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3c78d44c-9dfe-4812-977b-57fd7136ae97 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:32.310439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3c78d44c-9dfe-4812-977b-57fd7136ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:32.414072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 744/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:32.453102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:32.454744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:32.557754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd836f-510c-414b-947e-f95d2cf7ff9e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:32.637153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:32.639189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:32.639606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:32.639914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:32.640190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:32.641848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:32.754074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:32.755467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a722a2-2b5a-48a0-9b9e-b75a0e9bacd5 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:32.773925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 743/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:34.988020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:34.988598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:34.988598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:34.989933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 745/1488] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:34.995289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03ef1a-0b7f-4969-b5f0-3d9fe62a0fd2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:34.996329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03ef1a-0b7f-4969-b5f0-3d9fe62a0fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:34.996447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03ef1a-0b7f-4969-b5f0-3d9fe62a0fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:35.064089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2f03ef1a-0b7f-4969-b5f0-3d9fe62a0fd2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:35.065278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f03ef1a-0b7f-4969-b5f0-3d9fe62a0fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:35.117395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:35.162130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 744/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:35.203619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:35.205133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:35.242826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:35.316392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d466d0-02a4-4b95-9b38-b6cb695e8ac1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:35.391318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:35.391962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:35.392246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:35.392522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:35.394159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:35.484099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:35.513263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:35.514648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4cd4c-1eb7-4b82-bdf9-7fff2dc638c2 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:35.543131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 746/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:40:38.034204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:38.034780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:38.034780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:38.036171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 745/1491] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 21 10:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:40:38.041518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2495f-0f49-4344-8ba2-ae50349d22b9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:38.041968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2495f-0f49-4344-8ba2-ae50349d22b9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:38.042074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2495f-0f49-4344-8ba2-ae50349d22b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:38.106770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4da2495f-0f49-4344-8ba2-ae50349d22b9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:40:38.107980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4da2495f-0f49-4344-8ba2-ae50349d22b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:38.209086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 747/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 21 10:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:40:38.249503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:40:38.251067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:38.360338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229cada-a7c9-44f1-abb2-1cfd8c029c64 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:40:38.438283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:40:38.438960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:40:38.439239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:40:38.439509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:40:38.441166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:40:38.451830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:38.557820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:40:38.560262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fa69f-f273-4ca6-92d6-5f8faae3aa95 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:40:38.579007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 746/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:40:48.207259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=82 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:40:48.576588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=50 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:20.639451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:20.640036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:20.640036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:20.719503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:42:20.720670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:20.819971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 748/1494] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:42:20.828414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:20.828973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:20.829079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:20.917700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:42:20.918952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:21.025535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 747/1495] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:42:21.026691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:21.027990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:21.039896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074281b0-a529-4c18-ba77-5fa2db27e2dc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:21.107291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:21.144604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:21.145267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:21.145544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:21.145812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:21.146526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:42:21.146863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:21.147872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:21.147872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:42:21.147872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:42:21.149095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:21.160146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 749/1496] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:42:21.169029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:21.178201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682eb54-bcd5-492a-ad93-4333f3533d90 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:21.249698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:21.288679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:21.290021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:21.290313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:21.290617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:21.291340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:21.293165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:21.293165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:42:21.293165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:42:21.294391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:21.415555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 748/1497] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:42:21 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:42:21.419837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:21.421419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:21.455999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bf446-db67-4d7e-8515-14056816f525 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:21.552854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:21.553439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:21.553720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:21.553992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:21.555448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:21.556547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:21.556547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:42:21.556547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:42:21.557757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:21.694224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 750/1498] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:42:21 2022] POST /identity/v3/users => generated 289 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:42:21.699019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:21.700548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:21.766870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abad10c-a2ff-498f-a729-c38125488265 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:21.854143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:21.865021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:21.865702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:21.865975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:21.866244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:21.866999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:21.868971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:21.868971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:42:21.868971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:42:21.869857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:21.882054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 749/1499] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:42:21.888652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:21.890211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:21.901901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed121d-ad6f-4b37-b4e4-a37bbc49d770 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:21.926439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:21.997618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:21.998286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:21.998596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:21.998879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:21.999634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:22.001148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:22.001148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:42:22.001148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:42:22.002050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc02cea-a22f-432c-910e-982091c7be9a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:22.015257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 751/1500] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:42:22.019575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:22.021133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:22.056518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f2f7-ebf5-43db-8610-a0acc5393e3e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:22.158292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:22.159216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.159496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.159797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] PATH_INFO: `/v3/projects/4b892382991d45818dd17b2be6c9e39d/users/9f7b46024e7c4e6e9c91e35e3d6cf278/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.182231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b892382991d45818dd17b2be6c9e39d, user_id=9f7b46024e7c4e6e9c91e35e3d6cf278, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:22.183301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:22.183301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:42:22.183301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:42:22.185228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddbaf5-e873-4dfb-8890-3af7e276e675 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:22.259780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 750/1501] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:42:22 2022] PUT /identity/v3/projects/4b892382991d45818dd17b2be6c9e39d/users/9f7b46024e7c4e6e9c91e35e3d6cf278/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:42:22.271937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348d9e-5fa4-4fc6-b35e-c4f5234f1775 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.272223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348d9e-5fa4-4fc6-b35e-c4f5234f1775 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.272499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348d9e-5fa4-4fc6-b35e-c4f5234f1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.329792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-04348d9e-5fa4-4fc6-b35e-c4f5234f1775 None None] MFA Rules not processed for user `9f7b46024e7c4e6e9c91e35e3d6cf278`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:42:22.331012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-04348d9e-5fa4-4fc6-b35e-c4f5234f1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:22.411759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 752/1502] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:42:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:42:22.426160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.426453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.426755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.428162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 751/1503] 10.222.0.48 () {54 vars in 964 bytes} [Thu Jul 21 10:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:42:22.433168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a35857-98b2-4ba5-9d48-94c8d6676c88 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.433453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a35857-98b2-4ba5-9d48-94c8d6676c88 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.433728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a35857-98b2-4ba5-9d48-94c8d6676c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.465780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:22.511579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a7a35857-98b2-4ba5-9d48-94c8d6676c88 None None] MFA Rules not processed for user `1ec420ff94214b3ab6a041395c0a491c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:42:22.512776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a7a35857-98b2-4ba5-9d48-94c8d6676c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:22.588025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 753/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Jul 21 10:42:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:42:22.595672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:22.597201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:22.653433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:22.713482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0491f5-bbf0-4335-83f0-8b6403c77f55 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:22.791040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:22.791698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.791984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.792255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.793916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:22.902803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:22.904246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:22.907765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:22.924553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 752/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:42:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:42:22.967156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542d8b-7cf2-4009-b59a-64195533c57e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:22.967602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542d8b-7cf2-4009-b59a-64195533c57e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:22.967696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9542d8b-7cf2-4009-b59a-64195533c57e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:22.969179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 754/1506] 10.222.0.48 () {52 vars in 829 bytes} [Thu Jul 21 10:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:42:22.972785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:22.974068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:23.076793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3871c-93eb-4c4d-a441-da3368ca33c6 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:23.178086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:23.178916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:23.179203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:23.179481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] PATH_INFO: `/v3/users/9f7b46024e7c4e6e9c91e35e3d6cf278` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:23.192829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7b46024e7c4e6e9c91e35e3d6cf278)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:23.193866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:23.193866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:42:23.193866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:42:23.195118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:23.205329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:23.419347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7b46024e7c4e6e9c91e35e3d6cf278'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:23.432541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7b46024e7c4e6e9c91e35e3d6cf278'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:23.509591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7b46024e7c4e6e9c91e35e3d6cf278'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:23.510124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7b46024e7c4e6e9c91e35e3d6cf278'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:23.510566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7b46024e7c4e6e9c91e35e3d6cf278'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:23.536624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 753/1507] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:42:22 2022] DELETE /identity/v3/users/9f7b46024e7c4e6e9c91e35e3d6cf278 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:42:23.541193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:23.541493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:23.541772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:23.600801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:42:23.603666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:23.682224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:23.694293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 755/1508] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:42:23.705154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:23.706404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:23.811914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d8da-ef78-4010-ada2-4a16ff795491 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:23.871991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:23.893081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:23.900998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:23.900998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:23.900998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:23.900998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:24.138729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:24.140224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587822a-75ac-4bc1-9ea5-34f007d7e95a service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:24.159561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 754/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:42:24.676233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:42:24.677539 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:42:24.691262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc5712-f9ac-43af-ba44-94f2246d337d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:42:24.804508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:42:24.806508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:42:24.806814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:42:24.807084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] PATH_INFO: `/v3/projects/4b892382991d45818dd17b2be6c9e39d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:42:24.817255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b892382991d45818dd17b2be6c9e39d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:42:24.818281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:42:24.818281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:42:24.818281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:42:24.819529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:42:24.938684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b892382991d45818dd17b2be6c9e39d', 'request_id': 'req-40bb8c49-07ca-4440-b8d9-749a2fb521da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:25.006301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b892382991d45818dd17b2be6c9e39d', 'request_id': 'req-40bb8c49-07ca-4440-b8d9-749a2fb521da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:25.022250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b892382991d45818dd17b2be6c9e39d', 'request_id': 'req-40bb8c49-07ca-4440-b8d9-749a2fb521da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:25.047294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] The token cache is being invalidate because project 4b892382991d45818dd17b2be6c9e39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:42:25.047581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:25.048106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-40bb8c49-07ca-4440-b8d9-749a2fb521da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:42:25.049532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 756/1510] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:42:24 2022] DELETE /identity/v3/projects/4b892382991d45818dd17b2be6c9e39d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:42:34.157104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=62 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:42:35.045200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:44:08.796841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:08.797408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:08.797408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:08.798692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:08.803557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e4da-cb5e-4982-b7be-1c86fe573f4a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:08.804020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e4da-cb5e-4982-b7be-1c86fe573f4a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:08.804127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e4da-cb5e-4982-b7be-1c86fe573f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:08.884582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e200e4da-cb5e-4982-b7be-1c86fe573f4a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:08.885777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e200e4da-cb5e-4982-b7be-1c86fe573f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:08.989029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:09.150847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:09.152956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:09.276855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae27f88-2b2e-488f-909c-126e67aeda4f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:09.283011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:09.283657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:09.283943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:09.284221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:09.292580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:09.294217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc3f7-77ea-4b22-9e08-42a04e5b14b6 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:09.303918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 756/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:10.146933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203f07-aec2-463c-8d89-c134cccb5f29 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:10.147489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203f07-aec2-463c-8d89-c134cccb5f29 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:10.147489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203f07-aec2-463c-8d89-c134cccb5f29 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:10.148871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:10.155074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752de68-520c-4840-8a9e-c5ef0685e3b8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:10.155649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752de68-520c-4840-8a9e-c5ef0685e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:10.155649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752de68-520c-4840-8a9e-c5ef0685e3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:10.174922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d752de68-520c-4840-8a9e-c5ef0685e3b8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:10.176150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d752de68-520c-4840-8a9e-c5ef0685e3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:10.184413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:10.311159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ef1-23fa-4c0e-aca7-3168a265ca47 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:10.311445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ef1-23fa-4c0e-aca7-3168a265ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:10.311727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5ef1-23fa-4c0e-aca7-3168a265ca47 None None] PATH_INFO: `/v3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:10.313029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 759/1516] 10.222.0.57 () {54 vars in 925 bytes} [Thu Jul 21 10:44:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:44:10.324148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8466-71ce-408d-a849-8c0a660e9bc4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:10.324508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8466-71ce-408d-a849-8c0a660e9bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:10.324850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8466-71ce-408d-a849-8c0a660e9bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:10.347278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5ef1-23fa-4c0e-aca7-3168a265ca47 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:10.362442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5ef1-23fa-4c0e-aca7-3168a265ca47 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:10.367867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:10.368503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:10.368788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:10.369066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:10.378203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:10.379829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539963-2088-49fe-abb4-89aba7105996 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:10.390127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 760/1517] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:44:10.394234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-97af8466-71ce-408d-a849-8c0a660e9bc4 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:10.395727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-97af8466-71ce-408d-a849-8c0a660e9bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:10.456324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:44:10.476469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 758/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 21 10:44:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:19.609296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fff3d-8fb5-413e-8660-30c21106a33a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:19.609875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fff3d-8fb5-413e-8660-30c21106a33a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:19.609875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fff3d-8fb5-413e-8660-30c21106a33a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:19.611177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 761/1519] 10.222.0.48 () {54 vars in 961 bytes} [Thu Jul 21 10:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:19.616392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7861ef-f6d2-4d04-9d0c-467e7ba33cc2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:19.616892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7861ef-f6d2-4d04-9d0c-467e7ba33cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:19.617048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7861ef-f6d2-4d04-9d0c-467e7ba33cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:19.635098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5e7861ef-f6d2-4d04-9d0c-467e7ba33cc2 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:19.636312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5e7861ef-f6d2-4d04-9d0c-467e7ba33cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:19.645584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 759/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Jul 21 10:44:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:19.654442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fff3d-8fb5-413e-8660-30c21106a33a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:19.667993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fff3d-8fb5-413e-8660-30c21106a33a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:19.673345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:19.673991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:19.674279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:19.674594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:19.684603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:19.686194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611486a5-3057-411a-b4c8-4453a5d9b9ab service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:19.697446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 762/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:21.639109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3803e-7b7b-4a8d-b940-f8efd62c9694 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:21.639643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3803e-7b7b-4a8d-b940-f8efd62c9694 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:21.639643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3803e-7b7b-4a8d-b940-f8efd62c9694 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:21.640950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:21.646262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f697-b4f2-49cf-8e83-ef8de8b1b141 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:21.646787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f697-b4f2-49cf-8e83-ef8de8b1b141 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:21.646884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f697-b4f2-49cf-8e83-ef8de8b1b141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:21.667765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e914f697-b4f2-49cf-8e83-ef8de8b1b141 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:21.669188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e914f697-b4f2-49cf-8e83-ef8de8b1b141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:21.677230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:44:21.833080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3803e-7b7b-4a8d-b940-f8efd62c9694 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:21.848420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3803e-7b7b-4a8d-b940-f8efd62c9694 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:21.854185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:21.854857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:21.855136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:21.855404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:21.865287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:21.866904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29646e-d103-4241-8365-0c29f21abec3 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:21.877388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 761/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:22.704814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a33c-c77e-48ef-889e-32b67f9a8f4c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:22.705385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a33c-c77e-48ef-889e-32b67f9a8f4c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:22.705385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a33c-c77e-48ef-889e-32b67f9a8f4c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:22.706757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 10:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:22.711792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1040cf-88a3-4655-b91b-f9e5c786af36 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:22.712232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1040cf-88a3-4655-b91b-f9e5c786af36 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:22.712355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1040cf-88a3-4655-b91b-f9e5c786af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:22.732223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8e1040cf-88a3-4655-b91b-f9e5c786af36 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:22.733412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8e1040cf-88a3-4655-b91b-f9e5c786af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:22.741218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:22.893937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7a33c-c77e-48ef-889e-32b67f9a8f4c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:22.909395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7a33c-c77e-48ef-889e-32b67f9a8f4c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:22.914903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:22.915543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:22.915828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:22.916102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:22.925579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:22.927281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73806-88c3-49dc-b57e-02b8bfb280a7 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:22.938250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 765/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:26.900020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:26.900542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:26.900778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:26.902323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:26.908388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16576c-0cc1-4c50-aaab-dea8e891e950 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:26.908878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16576c-0cc1-4c50-aaab-dea8e891e950 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:26.909012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16576c-0cc1-4c50-aaab-dea8e891e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:26.927452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0b16576c-0cc1-4c50-aaab-dea8e891e950 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:26.928636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0b16576c-0cc1-4c50-aaab-dea8e891e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:26.937262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:27.015680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:27.017438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:27.100689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415c3e0-0720-4582-88fc-4f48b51249eb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:27.104205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:27.104848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:27.105128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:27.105401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:27.114252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:27.115747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1fc8d-9f63-435e-bb2b-1db115a42bc8 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:27.125223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 764/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:29.207339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5a54c-a939-445a-b629-9cda56dcba41 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:29.207909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5a54c-a939-445a-b629-9cda56dcba41 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:29.207909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5a54c-a939-445a-b629-9cda56dcba41 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:29.209255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:29.214355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377e7b-674a-44de-943a-ba1c0f5add9c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:29.214788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377e7b-674a-44de-943a-ba1c0f5add9c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:29.214933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377e7b-674a-44de-943a-ba1c0f5add9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:29.234648 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e6377e7b-674a-44de-943a-ba1c0f5add9c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:29.235832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e6377e7b-674a-44de-943a-ba1c0f5add9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:29.243657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:29.281248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5a54c-a939-445a-b629-9cda56dcba41 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:29.292222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5a54c-a939-445a-b629-9cda56dcba41 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:29.296564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:29.297198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:29.297480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:29.297757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:29.306682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:29.308105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950379b-242a-4db9-9f82-87e5ea499585 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:29.318156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 768/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:32.800877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:32.801457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:32.801457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:32.802766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:32.808058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7920-50f3-4876-8094-78df4ef0f552 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:32.812135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7920-50f3-4876-8094-78df4ef0f552 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:32.812135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7920-50f3-4876-8094-78df4ef0f552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:32.815094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:44:32.827649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-79fa7920-50f3-4876-8094-78df4ef0f552 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:32.828852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-79fa7920-50f3-4876-8094-78df4ef0f552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:32.837127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:33.146573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:33.148403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:33.231466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7f556-08de-4921-9bf4-40f8e02ddecc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:33.235202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:33.235876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:33.236156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:33.236431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:33.245724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:33.248179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a3cb-7361-4521-b5a1-3601c61c85de service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:33.259889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 767/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:35.398967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:35.399570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:35.399570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:35.400915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 10:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:44:35.406921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe033-d19d-485f-82c8-67c2b28f1a5b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:35.407414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe033-d19d-485f-82c8-67c2b28f1a5b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:35.407522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe033-d19d-485f-82c8-67c2b28f1a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:35.425883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7d6fe033-d19d-485f-82c8-67c2b28f1a5b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:44:35.427029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7d6fe033-d19d-485f-82c8-67c2b28f1a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:35.436073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:44:35.748510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:44:35.751999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:44:35.767132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81062-d65e-4e29-aaa5-3c831190e56e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:44:35.774807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:44:35.774807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:44:35.774807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:44:35.774807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:44:35.782571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:44:35.784067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa18174-2051-4375-909d-00625690bc7a service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:44:35.795081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 771/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:44:45.421081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:44:45.789948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:06.572377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.573308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.573308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.614579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:06.615948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:06.631479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 769/1540] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:06.641773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734b2b-8eb9-4656-8051-4ca18b1ec454 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.642413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734b2b-8eb9-4656-8051-4ca18b1ec454 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.642413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734b2b-8eb9-4656-8051-4ca18b1ec454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.671738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dc734b2b-8eb9-4656-8051-4ca18b1ec454 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:06.673038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dc734b2b-8eb9-4656-8051-4ca18b1ec454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:06.682592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 772/1541] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:06.687454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:06.700633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fa8a1-093f-4342-ac90-840f7a984632 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:06.710845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:06.711536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.711819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.712092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.713510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:06.713965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:06.715023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:06.715023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:46:06.715023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:46:06.715908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:06.732550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 770/1542] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:06.736131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7196e9-4eaa-40e2-b39b-455252c568c6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.736131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7196e9-4eaa-40e2-b39b-455252c568c6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.736131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7196e9-4eaa-40e2-b39b-455252c568c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.767892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:06.775695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4a7196e9-4eaa-40e2-b39b-455252c568c6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:06.777036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4a7196e9-4eaa-40e2-b39b-455252c568c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:06.781579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7f871-b4f9-4926-92b0-eb7184716174 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:06.788965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:06.793671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.793671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.793671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.793671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:06.793671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:06.799856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 773/1543] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:06.811001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.816529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.816529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.850076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:06.851651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:06.870940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 774/1544] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:06.876707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:06.890069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c7afd-26b0-4a23-a794-c1fbc443c80f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:06.903961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:06.908249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:06.908249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:06.908249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:06.908249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:06.908249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:06.909400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:06.929797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 775/1545] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:46:06.945509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:06.953608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 771/1546] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/projects => generated 393 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:06.957817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:06.964952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc182-33ff-4ac3-8269-cbd3703c7106 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.060210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.060888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.061170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.061452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.062146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.063209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:07.063209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:46:07.063209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:46:07.064412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160b37c-fb09-4d8f-a4ed-c5a146ff771d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.097106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c72687-507f-4a92-81f8-cefab9087ca1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.106960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.107571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.107857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.108259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.108976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.110009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:07.110009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:46:07.110009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:46:07.111274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adda4f-c9ce-4a46-ae8c-28d8940173ac admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.196387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 776/1547] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:07.198599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18708-2c87-4659-92a5-ed42ba445989 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.198887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18708-2c87-4659-92a5-ed42ba445989 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.199170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18708-2c87-4659-92a5-ed42ba445989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.218042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-25e18708-2c87-4659-92a5-ed42ba445989 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.219291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-25e18708-2c87-4659-92a5-ed42ba445989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.223991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 772/1548] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:06 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:07.225719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd50f-8915-44c3-bac0-ddf55fa15d40 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.225719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd50f-8915-44c3-bac0-ddf55fa15d40 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.225977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd50f-8915-44c3-bac0-ddf55fa15d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.246356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bc7bd50f-8915-44c3-bac0-ddf55fa15d40 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.247597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bc7bd50f-8915-44c3-bac0-ddf55fa15d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.286130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 777/1549] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.288501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1955a6-465c-4e50-b18b-e375913fd37a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.288501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1955a6-465c-4e50-b18b-e375913fd37a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.288501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1955a6-465c-4e50-b18b-e375913fd37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.309101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa1955a6-465c-4e50-b18b-e375913fd37a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.310319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa1955a6-465c-4e50-b18b-e375913fd37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.318492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 778/1550] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.319859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.320151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.320428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.320779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 773/1551] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.322650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cef1d-6f03-43ab-bf9e-e98f6b13d514 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.322944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cef1d-6f03-43ab-bf9e-e98f6b13d514 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.323222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cef1d-6f03-43ab-bf9e-e98f6b13d514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.342425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b83cef1d-6f03-43ab-bf9e-e98f6b13d514 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.342715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.343660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b83cef1d-6f03-43ab-bf9e-e98f6b13d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.343787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.351931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 774/1552] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.353569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.353961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.354241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.359444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 779/1553] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.360099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.370356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23e844-0115-4568-9936-472e06cbb85a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.374652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.375932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.376144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.376831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.377117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.377409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.378093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.380046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6997d8-5c23-4969-8561-55a3911fe25a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.384085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 775/1554] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.385619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.398433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91497-97f2-468f-aba4-afd8b775f222 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.404293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.404973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.405255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.405529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.406255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.407891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc7c58-4dc7-4c0c-bc6d-d613e7e36123 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.418369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 776/1555] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:07.419800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285f5e-e621-4b34-8b2a-99641e6f4226 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.420092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285f5e-e621-4b34-8b2a-99641e6f4226 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.420370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285f5e-e621-4b34-8b2a-99641e6f4226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.439032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-14285f5e-e621-4b34-8b2a-99641e6f4226 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.443242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-14285f5e-e621-4b34-8b2a-99641e6f4226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.449088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 777/1556] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.450457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb24766-7110-4084-beea-d98a7ba35b5b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.450771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb24766-7110-4084-beea-d98a7ba35b5b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.451047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb24766-7110-4084-beea-d98a7ba35b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.473634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 780/1557] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:07.473868 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0eb24766-7110-4084-beea-d98a7ba35b5b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.473983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481bce0-4f27-4407-af41-143b2fb33bc0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.473983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481bce0-4f27-4407-af41-143b2fb33bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.473983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481bce0-4f27-4407-af41-143b2fb33bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.474260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0eb24766-7110-4084-beea-d98a7ba35b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.480533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 778/1558] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.481891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644381ac-6ae6-4070-a8d8-2031084e5b03 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.482186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644381ac-6ae6-4070-a8d8-2031084e5b03 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.482451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644381ac-6ae6-4070-a8d8-2031084e5b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.489279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:07.495969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4481bce0-4f27-4407-af41-143b2fb33bc0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.497683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4481bce0-4f27-4407-af41-143b2fb33bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.503091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-644381ac-6ae6-4070-a8d8-2031084e5b03 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.504265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-644381ac-6ae6-4070-a8d8-2031084e5b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.505795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 781/1559] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.507204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.507496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.507776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.513407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 779/1560] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.514921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.515062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.515252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.527469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.528692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.533152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:07.535071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:07.537039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 782/1561] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.539504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.542295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 780/1562] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:07.543206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.550865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232ce90-e10d-4380-9689-5efab0d434fe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.552846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffe2d6-998c-4e89-802a-f2ab4ac43140 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.556477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.557161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.557610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.557900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.558731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.563635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.563635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.563635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.563962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.564070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.564070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.564070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.564337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:07.564431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.575377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 783/1563] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:07.575377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.579264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 781/1564] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.580112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.590187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60bc6a-894c-4fab-8ef8-30b4e9905061 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.591289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9892-e5d1-445f-9f37-3d6918fbcd22 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.594828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.595485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.595770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.595942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.596080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.596587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.596747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.596860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.597128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.597846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.598411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.598764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.600013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.612606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 784/1565] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:07.612606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.620980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 782/1566] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.620980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.621713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561312d7-0866-46e8-b86e-26301c83eaa6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.626245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.626928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.627266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.627551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.628275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.628580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.630174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.631547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732247d-b4a4-4cba-8760-3edffd03a8a9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.636406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.637077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.637361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.637799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.638571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.638907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.640695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.642920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 785/1567] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.643852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.654314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba36a7-63f1-4a1f-a0fc-e65c61996386 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.661475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 783/1568] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.661475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.661723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.661723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.661723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.661723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.662087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.662087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.663108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.667148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b057d2-b772-4c47-b0f6-0a38ac8b2fa4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.673776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.674651 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.675102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.675387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.676122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:07.676372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 786/1569] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.676760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.677341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.678352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.688916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8375-11cc-4e63-8b2e-2006f509ed4e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.693254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 784/1570] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:07.694381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.695504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.695723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.696066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] PATH_INFO: `/v3/projects/2ff45af5bd82457b9fd080026ea65b5b/users/a2cb1079722947dfa37946c03b7297a3/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.702087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.712626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297509-fcf5-4ace-927a-cb879bcf5f8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.714305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff45af5bd82457b9fd080026ea65b5b, user_id=a2cb1079722947dfa37946c03b7297a3, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.715403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:07.715403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:46:07.715403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:46:07.717360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.717732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.718499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.718747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.718999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.719693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.721748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.826174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 787/1571] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:07 2022] PUT /identity/v3/projects/2ff45af5bd82457b9fd080026ea65b5b/users/a2cb1079722947dfa37946c03b7297a3/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:07.827226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.869666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 785/1572] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:07.869666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.903233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11e720-2881-4b7f-9d4e-479b94e1f2e5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.937460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49b82-36c6-4688-a0b9-f549ffdef4d2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.942159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.942853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.943134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.943408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.944109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.946015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.964647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.965329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.965604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.965901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.966651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.968261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:07.979571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 788/1573] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:07.980527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:07.990166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc86aa3-7e9f-48ed-b676-a808b58582ff admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:07.995714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:07.996350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:07.996632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:07.996908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:07.997616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:07.999535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.030049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 786/1574] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.031814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.088687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 789/1575] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:07 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.089982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.097076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e0f9-f287-4487-b19e-75de67e22394 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.101240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:08.111667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:08.155549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.156114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0adbdc-d4b6-4fce-8045-296fa9964630 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.156319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.156665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.157037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.157894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.162724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.163335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.164133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.164415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.164688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.165954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.168336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc9edd-ca19-48e7-a221-563ebbfd4bd6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.282031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 787/1576] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/projects => generated 405 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.283186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.312805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 790/1577] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.312805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.312805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.312805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.328802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:08.330041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:08.356779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7551-466b-4c47-be7e-67344b691ce2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.391910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 791/1578] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:08.393564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.406938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839bb90-bb5f-410c-b207-053dcf8d2040 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.411592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.412278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.412557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.412835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.413529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.415495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.421016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.421691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.421972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.422242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.423206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.426290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.537375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 792/1579] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.538180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.548193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd1248-052a-4799-86e2-cf30f024f3dc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.552911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.553797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.554095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.554363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] PATH_INFO: `/v3/projects/9ceeabe5ea2a4a24b28901f50be5d532/users/ad7606b56fec4723b733f838ec34a17e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.562782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 788/1580] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/projects => generated 383 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.563721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.573863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ceeabe5ea2a4a24b28901f50be5d532, user_id=ad7606b56fec4723b733f838ec34a17e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.576556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.594006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:08.626835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 793/1581] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:08 2022] PUT /identity/v3/projects/9ceeabe5ea2a4a24b28901f50be5d532/users/ad7606b56fec4723b733f838ec34a17e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:08.626835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.632665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1162e-e9ae-480a-a0f9-4d6c79e968b7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.692745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1c96-d941-4e23-805a-b6e0587a4333 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.694101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.694835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.695169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.695391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.696088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.698404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.698611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.698882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.699162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.699433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.700117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.703593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.709878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:08.789226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 789/1582] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.790105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.801810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160e0e7-2efc-426b-ae01-a9b5b81f335f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.807385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.808253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.808537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.808812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.809505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.811457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.823276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 794/1583] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.823276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.829493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866c83-c8b8-44f2-86a6-e1a4ae1afc02 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.835877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.836552 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.836837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.837118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.838348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.840449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.932491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 795/1584] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.933503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.936187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 790/1585] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:08.937114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.947135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffb6c-e040-4236-8e89-a25ee41b2e24 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.952336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.953258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.953661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.954054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.955199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.957052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:08.969165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 791/1586] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:08.970079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:08.983528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9273d2-bf85-48e5-bd1d-fb17b483ad74 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:08.988934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:08.989608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:08.989920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:08.990192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:08.990926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:08.992844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.001942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9161-6233-4d65-b477-56c0c69ee282 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.005753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.006410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.006727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.007007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.016640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.018077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3aa90-b097-435f-a8fb-335c9bb286eb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.029085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 796/1587] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:09.030365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.030704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.031081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.049773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:09.051024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:09.059235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 797/1588] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:09.063814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.070150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991537-5b7b-419a-be9a-9652b2f56c99 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.076018 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.076671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.076954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.077232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.077961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.079600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.086050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 792/1589] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:08 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:09.089895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.093676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 798/1590] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.093676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.101347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268b959-3a33-4cf7-878c-4f73b7490bf3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.101662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d01be-bc69-4573-a827-2bf80ed6205e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.106959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.107122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.107549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.107652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.107834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.107968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.108109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.108227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.108825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.108933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.110420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.110558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.121565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 793/1591] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.122607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.124915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 799/1592] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.125786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.133128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639129-7ea1-411e-9223-7b558a4d0c27 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.135924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882002aa-cbbc-4063-8629-3ed488766cbe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.138262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.138971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.139253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.139531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.140261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.140680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.141345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.141641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.141856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.141964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.142621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.144376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.154222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 794/1593] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.155273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.162799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 800/1594] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.164224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.168851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c30c-60d8-4301-bb6d-b2fdec831b78 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.174036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.174749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.175110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.175365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.177630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc020d1-8281-4a9d-b821-7896eb82e85d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.185723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.186270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.186500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.186981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.187331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.191271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.191418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.191418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.200070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 795/1595] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:09.201133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.202146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 801/1596] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.202981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.214921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b155-fda2-488a-acc9-efa55582802c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.215960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550594d5-62fc-4700-a714-1cfa24247625 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.220293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.220796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.220948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.221290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.221571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.222064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.222288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.222421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.222800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.224131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.224245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.225528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.237955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 796/1597] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.238591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 802/1598] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.242110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.242265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.252913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de3def-adb4-4b2a-84a6-0988c7f55d36 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.253807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7d68-bb63-44a2-ad58-ec952b8b40ad admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.258195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.259187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.259574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.260380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.260959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.261317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.262137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.262416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.262782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.270657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.270798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.270798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.274088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 797/1599] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.274950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.281045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 803/1600] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.281045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.286743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eac9e-fa63-4b63-9dbf-d4ebba23e702 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.291851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b401c5-8a88-49dd-8a0b-cab64a163db6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.292225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.293113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.293397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.293671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] PATH_INFO: `/v3/projects/dfdeaccd863e4b6482f4e54c710b664b/users/7d16bd5326394f7aa5ff1a0a4704ebe4/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.296520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.297173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.297451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.297722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.298435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.302328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.310428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdeaccd863e4b6482f4e54c710b664b, user_id=7d16bd5326394f7aa5ff1a0a4704ebe4, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.313104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.315498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:09.316947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 804/1601] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:09.318149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.329336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d41f1-8eae-478d-93d2-320adb0d4afb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.335010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.335897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.336320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.336592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] PATH_INFO: `/v3/projects/0dc215aadb5345d791945658832aae4a/users/906e4bc4e9cc45ecb01cfca7918e461d/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.353603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc215aadb5345d791945658832aae4a, user_id=906e4bc4e9cc45ecb01cfca7918e461d, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.356406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.391972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 798/1602] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/dfdeaccd863e4b6482f4e54c710b664b/users/7d16bd5326394f7aa5ff1a0a4704ebe4/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.392224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.448210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 805/1603] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/0dc215aadb5345d791945658832aae4a/users/906e4bc4e9cc45ecb01cfca7918e461d/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.452538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.465638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc121a8-caaf-4d0d-b4a7-d55a70f0d224 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.491574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:09.537794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.538390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2662-64d4-4479-9942-429558132837 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.541077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.541077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.541077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] PATH_INFO: `/v3/projects/8158bf3416a04a538b87a7fab0f2cc23/users/f70cf1d6c0d748d38621b0be15c2336f/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.545126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.546289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.546799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.547390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] PATH_INFO: `/v3/projects/29c1b9c9f7024f8494e0a9b58fe35710/users/23b63d44bb304e97a38acc6d0b791c95/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.556444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8158bf3416a04a538b87a7fab0f2cc23, user_id=f70cf1d6c0d748d38621b0be15c2336f, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.559929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.570188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c1b9c9f7024f8494e0a9b58fe35710, user_id=23b63d44bb304e97a38acc6d0b791c95, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.573708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.661276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 799/1604] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/8158bf3416a04a538b87a7fab0f2cc23/users/f70cf1d6c0d748d38621b0be15c2336f/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.661276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.669956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 806/1605] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/29c1b9c9f7024f8494e0a9b58fe35710/users/23b63d44bb304e97a38acc6d0b791c95/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.669956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:09.747413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae3882-5ae0-4522-bbe7-85f2f395b622 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.758950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21290b2b-3545-4897-be29-dd06180c2a19 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:09.765585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.766769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.767158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.767603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] PATH_INFO: `/v3/projects/8f934391f10a47069cbfd9bfe3573e02/users/50d01be572d24e5db36c2863ab15f11a/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.786469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:09.788945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f934391f10a47069cbfd9bfe3573e02, user_id=50d01be572d24e5db36c2863ab15f11a, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.792789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350db4f-92df-445a-bbef-b576582a2387 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.820244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:09.821409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.821660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.821947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] PATH_INFO: `/v3/projects/38c890314ac34f0995bfc122aba43df3/users/eea41aaecfa341e78c43320ce3cac795/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.846279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c890314ac34f0995bfc122aba43df3, user_id=eea41aaecfa341e78c43320ce3cac795, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:09.850298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab2441-6f80-4acf-b4e3-38b3e1059d50 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:09.931667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 807/1606] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/8f934391f10a47069cbfd9bfe3573e02/users/50d01be572d24e5db36c2863ab15f11a/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.934308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777413-8fff-4ed2-bfb3-9932ded0dd17 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.934622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777413-8fff-4ed2-bfb3-9932ded0dd17 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.934900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777413-8fff-4ed2-bfb3-9932ded0dd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.950929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 800/1607] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:09 2022] PUT /identity/v3/projects/38c890314ac34f0995bfc122aba43df3/users/eea41aaecfa341e78c43320ce3cac795/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:09.954125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bf-624a-41b0-8b90-26f62bdb9618 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:09.954473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bf-624a-41b0-8b90-26f62bdb9618 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:09.954767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bf-624a-41b0-8b90-26f62bdb9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:09.975515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-75777413-8fff-4ed2-bfb3-9932ded0dd17 None None] MFA Rules not processed for user `7d16bd5326394f7aa5ff1a0a4704ebe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:09.976730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-75777413-8fff-4ed2-bfb3-9932ded0dd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:09.995939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-680203bf-624a-41b0-8b90-26f62bdb9618 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:09.997599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-680203bf-624a-41b0-8b90-26f62bdb9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.056561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 808/1608] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.059123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5f01-33a2-45c1-a71b-29558317f6cb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.059825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5f01-33a2-45c1-a71b-29558317f6cb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.060115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5f01-33a2-45c1-a71b-29558317f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.106692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 801/1609] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.107879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a4c46-44c2-4354-bce4-6b21d2d55963 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.108139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a4c46-44c2-4354-bce4-6b21d2d55963 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.108418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a4c46-44c2-4354-bce4-6b21d2d55963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.115020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-376d5f01-33a2-45c1-a71b-29558317f6cb None None] MFA Rules not processed for user `f70cf1d6c0d748d38621b0be15c2336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:10.116499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-376d5f01-33a2-45c1-a71b-29558317f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.146070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e52a4c46-44c2-4354-bce4-6b21d2d55963 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:10.147852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e52a4c46-44c2-4354-bce4-6b21d2d55963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.157088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 802/1610] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.158518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f12d2-e014-40d1-84fb-e91665233b21 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.158906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f12d2-e014-40d1-84fb-e91665233b21 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.159410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:10.172784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f12d2-e014-40d1-84fb-e91665233b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.210016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bd4f12d2-e014-40d1-84fb-e91665233b21 None None] MFA Rules not processed for user `50d01be572d24e5db36c2863ab15f11a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:10.222729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bd4f12d2-e014-40d1-84fb-e91665233b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.236205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 809/1611] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.237857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.238140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.238418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.270257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] MFA Rules not processed for user `eea41aaecfa341e78c43320ce3cac795`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:10.271669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.339161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 803/1612] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.341237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac47f-38eb-41be-b014-d1b5d8856b2c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.341762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac47f-38eb-41be-b014-d1b5d8856b2c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.342256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac47f-38eb-41be-b014-d1b5d8856b2c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.344814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 804/1613] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 21 10:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:10.346283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac47f-38eb-41be-b014-d1b5d8856b2c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.368600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 810/1614] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.368600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.368600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.433554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:10.443094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9474d8-087f-486f-b8b8-cdda2ea86ddf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.448284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.449477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.450250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.451030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.457467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac47f-38eb-41be-b014-d1b5d8856b2c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.470939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.472093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:10.474363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.476067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.476067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.480098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:10.490827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:10.494511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed8e6-1b10-45df-be9d-dcd350fe9766 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:10.494684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 811/1615] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:10.494684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.504277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 805/1616] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:10.504513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b000b5-69b4-48b8-abe3-64461053a736 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.508876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.509639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.510275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.510567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.510848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.511052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.511546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:10.511859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.513867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b15ec5-827b-4c31-8c6c-cda62be76d09 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:10.561075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:10.562406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:10.640859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 806/1617] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:10.641712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.652522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e07cf-22fa-4107-b66b-364cdc851be7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.657673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.658197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.658643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.658805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.659484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:10.661644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:10.678578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 812/1618] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/projects => generated 371 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:10.680093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4e-7830-4cee-80c4-928af740776c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.680384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4e-7830-4cee-80c4-928af740776c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.680702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4e-7830-4cee-80c4-928af740776c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.682782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 813/1619] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 21 10:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:10.683623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013a4e-7830-4cee-80c4-928af740776c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.778746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013a4e-7830-4cee-80c4-928af740776c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.785278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 807/1620] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:10.787176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:10.925240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.926344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.926728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.927008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.943518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6f72-0e27-410a-ac4c-7e8196d77753 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:10.949846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:10.950995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:10.951406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:10.951884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:10.953188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:10.955737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.024503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.026639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe3843-8244-4d96-9db4-ecae6eee0b3a service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.039511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 814/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:11.042024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.042538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.043031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.065767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:11.068382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:11.077242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 808/1622] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:10 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:11.078494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.092517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 815/1623] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 21 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:11.093401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.109747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e019d-56ff-4a0c-84e2-81116d533d27 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.122853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74864a-046b-4129-81fc-6baf535f0a83 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.126853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.127396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.127798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.127990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.128681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.134166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.135533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.136254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.136520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.136799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.137550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.139230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.166740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 816/1624] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:11.168055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.196992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0bbea-54fa-4610-b0b6-d47f5e8b33b7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.211632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.212308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.216505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.216505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.264271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 809/1625] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:11 2022] POST /identity/v3/users => generated 309 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:11.265005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.274991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:11.301816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539d44-527e-466e-b27b-ddf9d1aea57b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.313554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.315883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.316684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.316978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] PATH_INFO: `/v3/projects/9791569569ab4636a9777f0ca3d84808/users/1d3755aa8ccd42a49b5ce78c59e40a22/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.351946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.356149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791569569ab4636a9777f0ca3d84808, user_id=1d3755aa8ccd42a49b5ce78c59e40a22, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.358909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.363815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.387236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 817/1626] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:11.388378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.411625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140fc3-b5d9-4e9d-9648-bf416740c409 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.427166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.427911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.428200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.428481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.429224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.431271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.434889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:11.460355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 818/1627] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:11.460355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.477557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 810/1628] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:11 2022] PUT /identity/v3/projects/9791569569ab4636a9777f0ca3d84808/users/1d3755aa8ccd42a49b5ce78c59e40a22/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:11.479187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.505899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97a7fa-0da8-438f-828c-10d70b924582 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.629412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ef03-4a14-49b0-aa1c-8544ac50e359 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.643359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.644057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.644338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.644619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.645374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.647052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dc5cc-9a05-4545-8aab-f1c66cb10fcb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.671846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 811/1629] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:11.673366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f21bc-065b-4a28-ac83-bf9f018e11e8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.673664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f21bc-065b-4a28-ac83-bf9f018e11e8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.673945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f21bc-065b-4a28-ac83-bf9f018e11e8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.675809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 812/1630] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 21 10:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:11.677985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f21bc-065b-4a28-ac83-bf9f018e11e8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.705541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.706851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.707791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.707791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] PATH_INFO: `/v3/projects/19d269089cc940b5a44bd003fde971fa/users/f03f8ce9a5e243d39113b0b61458eaf4/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.715270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f21bc-065b-4a28-ac83-bf9f018e11e8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:11.725883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:11.727089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.727579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.728034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] PATH_INFO: `/v3/projects/9791569569ab4636a9777f0ca3d84808/users/1d3755aa8ccd42a49b5ce78c59e40a22/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.729414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d269089cc940b5a44bd003fde971fa, user_id=f03f8ce9a5e243d39113b0b61458eaf4, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.732606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb450c2-6241-4b22-9006-7426fe47e083 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.736105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791569569ab4636a9777f0ca3d84808, user_id=1d3755aa8ccd42a49b5ce78c59e40a22, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:11.739078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedc6ea-87f1-4ff5-8d60-af025204af4e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:11.781976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 813/1631] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:11 2022] PUT /identity/v3/projects/9791569569ab4636a9777f0ca3d84808/users/1d3755aa8ccd42a49b5ce78c59e40a22/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:11.783991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.784445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.784900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.810535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:11.812133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:11.828783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 819/1632] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:11 2022] PUT /identity/v3/projects/19d269089cc940b5a44bd003fde971fa/users/f03f8ce9a5e243d39113b0b61458eaf4/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:11.832133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe9053-af51-41f3-a507-f68372ff9339 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:11.832133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe9053-af51-41f3-a507-f68372ff9339 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:11.832133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe9053-af51-41f3-a507-f68372ff9339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:11.877102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d5fe9053-af51-41f3-a507-f68372ff9339 None None] MFA Rules not processed for user `1d3755aa8ccd42a49b5ce78c59e40a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:11.879861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5fe9053-af51-41f3-a507-f68372ff9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:11.952053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 814/1633] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:11.953635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:11.996787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 820/1634] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:12.003087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b9ae-28d8-4dae-9db7-3714e3a42b5f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.009925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:12.009925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b9ae-28d8-4dae-9db7-3714e3a42b5f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.009925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b9ae-28d8-4dae-9db7-3714e3a42b5f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.010576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 821/1635] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 21 10:46:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:12.011399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1b9ae-28d8-4dae-9db7-3714e3a42b5f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:12.069173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8370e-be3e-4ef3-8227-d1025614e141 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:12.116980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1b9ae-28d8-4dae-9db7-3714e3a42b5f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:12.122467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:12.124334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.124854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.125344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.215888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:12.216570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.217379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.217379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.217867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:12.221653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08d766-68fb-4eca-a0fa-173d7c3f8126 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:12.231183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:12.239249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 815/1636] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:12.245614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.245614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.245614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.267370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:12.269002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:12.309877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:12.313187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:12.346386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 822/1637] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:12.349510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:12.372563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7535c0-952f-490a-9865-6a5f0ec20369 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:12.382137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:12.384019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.385698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.386200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.417815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:12.420233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 816/1638] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:46:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:12.421114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:12.422379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:12.444792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 823/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:12.445507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3aa9-8e71-4a09-8cd9-723b98dee856 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:12.450390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:12.451328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.451606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.451878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] PATH_INFO: `/v3/projects/19d269089cc940b5a44bd003fde971fa/users/f03f8ce9a5e243d39113b0b61458eaf4/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.454283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d269089cc940b5a44bd003fde971fa, user_id=f03f8ce9a5e243d39113b0b61458eaf4, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:12.457299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdc297-ac85-48f5-aaf8-7d76bfa62ecf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:12.476881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 817/1640] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:12 2022] PUT /identity/v3/projects/19d269089cc940b5a44bd003fde971fa/users/f03f8ce9a5e243d39113b0b61458eaf4/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:12.486968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:12.490017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:12.505906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.506310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.506612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.521772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23ae8a-a838-47d2-af3a-d5941576c1e6 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:12.608661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] MFA Rules not processed for user `f03f8ce9a5e243d39113b0b61458eaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:12.612909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:12.699097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:12.899200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:12.899200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:12.899200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:12.902293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 818/1641] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:12.905496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:12.905496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:12.905496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 824/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:13.053999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:13.064867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e388c5-3e0a-4df8-bb90-d9b4de85ba7c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:13.069830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:13.070697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:13.070977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:13.071253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:13.095909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:13.100439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:13.120083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 819/1643] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:13.218766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:13.272914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:13.410946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162158-d31b-48a9-a136-119eb1ce4d65 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:13.418894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:13.422805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:13.423280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:13.423832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] PATH_INFO: `/v3/users/7d16bd5326394f7aa5ff1a0a4704ebe4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:13.444735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d16bd5326394f7aa5ff1a0a4704ebe4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:13.448136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:13.448136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jul 21 10:46:13.448136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: Jul 21 10:46:13.451844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:13.562865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:13.733914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:13.762248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce3d4-dd80-48c6-b0e6-3961c692bd91 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:13.807222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:13.808373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:13.810083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:13.810513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] PATH_INFO: `/v3/users/eea41aaecfa341e78c43320ce3cac795` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:13.822497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea41aaecfa341e78c43320ce3cac795)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:13.826853 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:13.828331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d16bd5326394f7aa5ff1a0a4704ebe4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:13.866792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d16bd5326394f7aa5ff1a0a4704ebe4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:13.870083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d16bd5326394f7aa5ff1a0a4704ebe4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:13.873889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d16bd5326394f7aa5ff1a0a4704ebe4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:13.982976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbacbc5-7b02-44b4-a847-376d1220d08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d16bd5326394f7aa5ff1a0a4704ebe4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.102668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 825/1644] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:13 2022] DELETE /identity/v3/users/7d16bd5326394f7aa5ff1a0a4704ebe4 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:14.114658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaec13-3c65-446e-a064-fcb4f716c781 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:14.115035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaec13-3c65-446e-a064-fcb4f716c781 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:14.115373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaec13-3c65-446e-a064-fcb4f716c781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:14.127195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea41aaecfa341e78c43320ce3cac795'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.140548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea41aaecfa341e78c43320ce3cac795'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.141356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4dbaec13-3c65-446e-a064-fcb4f716c781 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:14.142894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4dbaec13-3c65-446e-a064-fcb4f716c781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:14.222240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea41aaecfa341e78c43320ce3cac795'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.222240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea41aaecfa341e78c43320ce3cac795'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.223074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea41aaecfa341e78c43320ce3cac795'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:14.297814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 826/1646] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:14.300745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 820/1646] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:13 2022] DELETE /identity/v3/users/eea41aaecfa341e78c43320ce3cac795 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:14.305160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:14.305493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:14.307114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:14.331593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:14.332906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:14.373464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:14.381750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:14.383701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:14.474323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 827/1647] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:14.551740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6cc0-9aa7-4521-a26c-c0cf62a612ec admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:14.562089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:14.562948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:14.563307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:14.563658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:14.621380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:14.623562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:14.661389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 821/1648] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:14.907121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:14.908618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:14.924276 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f18ae3-a0de-4fb5-9fee-7ffff0ef4f9f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:14.933336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:14.934254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:14.934557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:14.934837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] PATH_INFO: `/v3/users/f70cf1d6c0d748d38621b0be15c2336f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:14.936218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f70cf1d6c0d748d38621b0be15c2336f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:14.938177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:15.052733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:15.066881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8ebad-8ebe-43fb-a906-bbf86fc815ef service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:15.081866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:15.082610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:15.082895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:15.083167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:15.107499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:15.108986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3fa4-03cf-47cd-bcdc-a1e323ed6835 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:15.139724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 822/1649] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:15.246720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f70cf1d6c0d748d38621b0be15c2336f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:15.268322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f70cf1d6c0d748d38621b0be15c2336f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:15.269965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f70cf1d6c0d748d38621b0be15c2336f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:15.271707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f70cf1d6c0d748d38621b0be15c2336f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:15.337087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f70cf1d6c0d748d38621b0be15c2336f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:15.374067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 828/1650] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:14 2022] DELETE /identity/v3/users/f70cf1d6c0d748d38621b0be15c2336f => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:15.379379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:15.380235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:15.380705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:15.424681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:15.426531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:15.526922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:15.540079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 823/1651] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:15.554996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:15.557470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:15.645172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c9bf-bcac-41be-ab32-3c23b25882fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:15.650942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:15.651618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:15.651904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:15.652186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:15.666128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:15.667978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:15.682522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 829/1652] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:16.686653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:16.689401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:16.719197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822330-5cab-4763-8e14-2ae5a594924c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:16.739760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:16.740434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:16.740947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:16.741344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] PATH_INFO: `/v3/projects/38c890314ac34f0995bfc122aba43df3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:16.742859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c890314ac34f0995bfc122aba43df3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:16.743961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 10:46:16.743961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: warnings.warn(msg) Jul 21 10:46:16.743961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: Jul 21 10:46:16.745539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:16.950217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c890314ac34f0995bfc122aba43df3', 'request_id': 'req-de35656f-97ef-48b7-b73f-4d33a0f6b379'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.112809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c890314ac34f0995bfc122aba43df3', 'request_id': 'req-de35656f-97ef-48b7-b73f-4d33a0f6b379'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.131035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c890314ac34f0995bfc122aba43df3', 'request_id': 'req-de35656f-97ef-48b7-b73f-4d33a0f6b379'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.154692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:17.156788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:17.234812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bf1e-c3e3-4ea0-be2e-f4d98d4b1faf service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:17.298867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] The token cache is being invalidate because project 38c890314ac34f0995bfc122aba43df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:17.300267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.304048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.309914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 824/1653] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:16 2022] DELETE /identity/v3/projects/38c890314ac34f0995bfc122aba43df3 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:17.319761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:17.332407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:17.464776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:17.558966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:17.564543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:17.565587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:17.565867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] PATH_INFO: `/v3/projects/dfdeaccd863e4b6482f4e54c710b664b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:17.568849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdeaccd863e4b6482f4e54c710b664b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:17.571034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:17.629175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35656f-97ef-48b7-b73f-4d33a0f6b379 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:17.639236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:17.640409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:17.640857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:17.641268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] PATH_INFO: `/v3/users/f03f8ce9a5e243d39113b0b61458eaf4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:17.642961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03f8ce9a5e243d39113b0b61458eaf4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:17.645954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:17.795137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdeaccd863e4b6482f4e54c710b664b', 'request_id': 'req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.954532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdeaccd863e4b6482f4e54c710b664b', 'request_id': 'req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.963229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdeaccd863e4b6482f4e54c710b664b', 'request_id': 'req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.986617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03f8ce9a5e243d39113b0b61458eaf4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.996189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] The token cache is being invalidate because project dfdeaccd863e4b6482f4e54c710b664b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:17.999534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:17.999534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfaa3cdc-a562-4bd9-90bf-77515c9fbc7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.011618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 830/1654] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:17 2022] DELETE /identity/v3/projects/dfdeaccd863e4b6482f4e54c710b664b => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:18.011771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03f8ce9a5e243d39113b0b61458eaf4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.015040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:18.015040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:18.015040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:18.062284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:18.068950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:18.080341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03f8ce9a5e243d39113b0b61458eaf4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.081699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03f8ce9a5e243d39113b0b61458eaf4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.082519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0e035aa9-e3b3-4c39-bb0a-b62955210bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03f8ce9a5e243d39113b0b61458eaf4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.125791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:18.149345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 825/1655] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:17 2022] DELETE /identity/v3/users/f03f8ce9a5e243d39113b0b61458eaf4 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:18.303783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 831/1656] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:18.308655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:18.308941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:18.309331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:18.365485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:18.366773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:18.483744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 826/1657] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:18.485234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:18.498604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c8e0-3d25-46bf-9f15-fd50c7c03dae None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:18.510139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:18.510871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:18.511158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:18.511437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:18.512180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:18.512487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:18.515990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:18.544976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 832/1658] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:18.548924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:18.568028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d86657-1efe-44b4-81ff-d8788cfe5254 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:18.592778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:18.593839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:18.594597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:18.595203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:18.596268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:18.597588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:18.603446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:18.605813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:18.625275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f13463-97ba-42fc-8b31-1ce013716944 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:18.646196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:18.647245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:18.647682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:18.648038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] PATH_INFO: `/v3/projects/8158bf3416a04a538b87a7fab0f2cc23` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:18.649448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8158bf3416a04a538b87a7fab0f2cc23)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:18.652475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:18.746009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 827/1659] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:18 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:18.748167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:18.835849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8158bf3416a04a538b87a7fab0f2cc23', 'request_id': 'req-8b6e0688-a9f6-46ad-8e34-f26911e9a007'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.919545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8524-29c9-4ea5-9bdf-457718991e86 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:18.937032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8158bf3416a04a538b87a7fab0f2cc23', 'request_id': 'req-8b6e0688-a9f6-46ad-8e34-f26911e9a007'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.949131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8158bf3416a04a538b87a7fab0f2cc23', 'request_id': 'req-8b6e0688-a9f6-46ad-8e34-f26911e9a007'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.982107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] The token cache is being invalidate because project 8158bf3416a04a538b87a7fab0f2cc23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:18.982390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.987755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:18.995800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 833/1660] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:18 2022] DELETE /identity/v3/projects/8158bf3416a04a538b87a7fab0f2cc23 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:19.000035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.004102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:19.091763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.092318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:19.093149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.095226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.095677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.096549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.100102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.168584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0688-a9f6-46ad-8e34-f26911e9a007 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.187503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.189616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.193885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.193885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] PATH_INFO: `/v3/users/1d3755aa8ccd42a49b5ce78c59e40a22` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.195606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3755aa8ccd42a49b5ce78c59e40a22)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.208851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.232522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 828/1661] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:18 2022] POST /identity/v3/users => generated 292 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:19.235716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.249628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:19.269890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002b-f671-4d2a-951e-7e2c8dfc159b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.277616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.279977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.280354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.280682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.282303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.291841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.319498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 829/1662] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:19.320781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.333326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ad1d-0927-451f-8afa-f40e5366f9df admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.342304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.348658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.348658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.348658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.348658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.348658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.371757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 830/1663] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:19.373391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.391616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1b05a-509b-4398-990d-46ae1cd52a63 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.401901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.404515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.406065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.406972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] PATH_INFO: `/v3/projects/0710a10e83c142349cd324ef9a421746/users/3c2ea0a1fbde4b0ca7d69269f173cb5e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.465522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0710a10e83c142349cd324ef9a421746, user_id=3c2ea0a1fbde4b0ca7d69269f173cb5e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.469001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.474207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3755aa8ccd42a49b5ce78c59e40a22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:19.487868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3755aa8ccd42a49b5ce78c59e40a22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:19.488642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3755aa8ccd42a49b5ce78c59e40a22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:19.489260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3755aa8ccd42a49b5ce78c59e40a22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:19.576504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 831/1664] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:19 2022] PUT /identity/v3/projects/0710a10e83c142349cd324ef9a421746/users/3c2ea0a1fbde4b0ca7d69269f173cb5e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:19.577687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.587369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:19.590099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fbbb3d0d-99f9-4f69-9617-ca3b68b52a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3755aa8ccd42a49b5ce78c59e40a22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:19.596537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:19.661858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 834/1665] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:18 2022] DELETE /identity/v3/users/1d3755aa8ccd42a49b5ce78c59e40a22 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:19.663809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.666908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.667418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.726274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] MFA Rules not processed for user `3c2ea0a1fbde4b0ca7d69269f173cb5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:19.727572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:19.759106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5e9a-0eba-444d-b1d5-acb3cc46739b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.861882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.863159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.863619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.864070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] PATH_INFO: `/v3/projects/19d269089cc940b5a44bd003fde971fa` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.866474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d269089cc940b5a44bd003fde971fa)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.872013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:19.887212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 835/1666] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:19.891796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:19.894866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:19.925839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce57e01-4d09-4997-b9ee-e6c28cbcff4f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:19.943035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:19.944536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:19.948028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:19.958673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:19.959839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:19.961952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.049110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d269089cc940b5a44bd003fde971fa', 'request_id': 'req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:20.082356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 836/1667] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:19 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:20.086389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.201974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ec74c-9741-4efb-beb1-7c9117f0bbce admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.205453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d269089cc940b5a44bd003fde971fa', 'request_id': 'req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:20.208288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.209302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.209846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.210599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.211611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.213189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d269089cc940b5a44bd003fde971fa', 'request_id': 'req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:20.214334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.235813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] The token cache is being invalidate because project 19d269089cc940b5a44bd003fde971fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:20.236109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:20.245709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-76cbb509-8c06-474b-8fe2-cd5810cc3b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:20.247743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 832/1668] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:19 2022] DELETE /identity/v3/projects/19d269089cc940b5a44bd003fde971fa => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:20.254109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4550c-fe76-4415-a3e4-e0d275fc465a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.254401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4550c-fe76-4415-a3e4-e0d275fc465a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.254773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4550c-fe76-4415-a3e4-e0d275fc465a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.289988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c3b4550c-fe76-4415-a3e4-e0d275fc465a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:20.291356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c3b4550c-fe76-4415-a3e4-e0d275fc465a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:20.308713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 833/1669] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:20.316920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.318652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.319811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.339715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 837/1670] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:20.344141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.347006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:20.351037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:20.352393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:20.366068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261aa4f-809d-46d7-8bc9-9f8d11918748 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.371508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.372412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.372826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.373242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.374091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 834/1671] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:20.374290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.376097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.377838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.401169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d538085-163f-490b-bea8-fd2fba7eeca5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.408367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.409990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.410525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.410933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 838/1672] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:46:20.411339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.412390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:20.412921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.416481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.418314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.434375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 835/1673] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:20.439227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.440278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea7a0f-be11-46ef-8173-1b7b74857a77 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.452724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.454098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673d9e-13a9-4e98-a84a-f8276dc09bc3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.456200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.456641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.457641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] PATH_INFO: `/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43/users/47ea1d8871944c90995edab42df0ed27/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.461016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.461900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.463095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.463493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.464432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.466959 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.489883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dbb38a56eb4190b76fcfd487f4af43, user_id=47ea1d8871944c90995edab42df0ed27, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.493157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.590907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 839/1674] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:20 2022] PUT /identity/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43/users/47ea1d8871944c90995edab42df0ed27/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:20.595060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.662269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 836/1675] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:20.667081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.707152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6fdad-191c-4144-b783-c94a41b4b038 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.784200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:20.790158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.792661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.794317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.794811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.799041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.812626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.832767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc85e78-1d33-4f21-896e-9d900e495133 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.844102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 840/1676] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:46:20.845061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.847956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:20.848926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.849739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.850492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.854501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.857273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.875618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9f3e4-2681-403e-ba31-226ff0f87260 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:20.894177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:20.896205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.897087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.897575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] PATH_INFO: `/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43/users/47ea1d8871944c90995edab42df0ed27/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.902337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dbb38a56eb4190b76fcfd487f4af43, user_id=47ea1d8871944c90995edab42df0ed27, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:20.907719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab15f-cfcd-42c9-acdf-ff8560000534 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:20.924944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 841/1677] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:20 2022] PUT /identity/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43/users/47ea1d8871944c90995edab42df0ed27/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:20.940748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:20.941360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:20.941860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:20.989667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 837/1678] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/users => generated 282 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:20.994438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:21.013898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] MFA Rules not processed for user `47ea1d8871944c90995edab42df0ed27`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:21.016622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503326a8-07ca-478f-b942-7a0688cbee43 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:21.017973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:21.028894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:21.201990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:21.206188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.206838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.207937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.220664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:21.226834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:21.247961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 842/1679] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:21.252854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:21.254827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:21.287278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 838/1680] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:21.292981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:21.322791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866bda1-b1bb-4cc6-be89-5fa51e9bc3a8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:21.342467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:21.343197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.348586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.348586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.350037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:21.355452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:21.378590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 839/1681] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:21.381629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:21.396523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c42e9-c131-493a-a6d7-af4e9e3b177a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:21.411883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:21.414564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc44b55-0b21-484b-b418-e7b2e799965c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:21.416500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.416789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.417127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.426301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:21.427666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.427666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.427837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] PATH_INFO: `/v3/projects/3503902c27b8436ea380375798e96d9a/users/a9c99f002ea84c7d8127b15e2f9a6406/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.449807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:21.453203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9d4a4-53ca-491c-b4ab-32851ea5e946 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:21.468916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3503902c27b8436ea380375798e96d9a, user_id=a9c99f002ea84c7d8127b15e2f9a6406, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:21.475318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:21.477969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 843/1682] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:21.648420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 840/1683] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:21 2022] PUT /identity/v3/projects/3503902c27b8436ea380375798e96d9a/users/a9c99f002ea84c7d8127b15e2f9a6406/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:21.660641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.661159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.661229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.695699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:21.768701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] MFA Rules not processed for user `a9c99f002ea84c7d8127b15e2f9a6406`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:21.770005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:21.867953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3575-a252-42f8-aad6-9c01c130c198 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:21.899485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:21.900876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:21.901164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:21.901439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:21.915245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:21.992801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 844/1684] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:21 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:21.994478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:22.152189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:22.154319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:22.180954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 841/1685] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:22.232166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf64e2-0211-415b-bfa0-8b229daa1290 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:22.257344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:22.258042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:22.258327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:22.258629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:22.262090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:22.264070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:22.294682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:22.295965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:22.336703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2c6fc-0666-45c7-8661-355737958ee9 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:22.362830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:22.374324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:22.376819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:22.377791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:22.384990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] PATH_INFO: `/v3/projects/9791569569ab4636a9777f0ca3d84808` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:22.387105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9791569569ab4636a9777f0ca3d84808)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:22.391796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:22.445033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 845/1686] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:21 2022] POST /identity/v3/projects => generated 365 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:22.455795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:22.591931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:22.717979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c5f9a-5f08-49bc-9801-f47d957199e0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:22.764774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9791569569ab4636a9777f0ca3d84808', 'request_id': 'req-7909d38f-b859-4898-8f7b-7a54f0eb109a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:22.874288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9791569569ab4636a9777f0ca3d84808', 'request_id': 'req-7909d38f-b859-4898-8f7b-7a54f0eb109a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:22.893798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9791569569ab4636a9777f0ca3d84808', 'request_id': 'req-7909d38f-b859-4898-8f7b-7a54f0eb109a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:22.925437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] The token cache is being invalidate because project 9791569569ab4636a9777f0ca3d84808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:22.926025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:22.927354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7909d38f-b859-4898-8f7b-7a54f0eb109a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:22.930172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 842/1687] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:22 2022] DELETE /identity/v3/projects/9791569569ab4636a9777f0ca3d84808 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:22.937152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca211089-91d8-4fe7-a018-7594b01e7e26 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:22.937707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca211089-91d8-4fe7-a018-7594b01e7e26 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:22.938144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca211089-91d8-4fe7-a018-7594b01e7e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:22.968995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ca211089-91d8-4fe7-a018-7594b01e7e26 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:22.970330 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ca211089-91d8-4fe7-a018-7594b01e7e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:22.974575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:22.976166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:22.976458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:22.977164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:22.977731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:22.984893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:22.985650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 843/1688] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:22.994832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:22.995481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:22.996157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.025158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:23.027079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:23.072793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 844/1689] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:23.081477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.102457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97748f43-820d-4363-9dcc-9a5f53047929 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.110235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.112448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.112755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.113327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.117697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:23.118139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.120894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.145254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 846/1690] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:22 2022] POST /identity/v3/users => generated 289 bytes in 686 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:23.150840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.153750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:23.170335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 845/1691] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:23.212279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.230136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313284-900d-4fe2-87ea-ad85fd4547aa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.246396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565594f-1713-482f-b96a-61a383ffb655 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.248912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.249592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.257842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.258731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.261637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.262623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.264913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.270560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.271851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.273201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.280822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.296495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.326728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 847/1692] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:23.332105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.362949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2b22-faef-4f80-8a5d-13c744e24b17 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.405039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.406946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.409044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 846/1693] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:23 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:23.410133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.411388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] PATH_INFO: `/v3/projects/06def8463fc840cdbd407353e1f48362/users/b720a53a8d2f4402b6ae3fffbc282182/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.417964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.484873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06def8463fc840cdbd407353e1f48362, user_id=b720a53a8d2f4402b6ae3fffbc282182, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.491730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.613375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:23.619703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 848/1694] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:23 2022] PUT /identity/v3/projects/06def8463fc840cdbd407353e1f48362/users/b720a53a8d2f4402b6ae3fffbc282182/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:23.625487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.735569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983959e-d6a0-412c-82d4-d7bf9edba242 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.841829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37268b63-b243-43db-8f66-45b2a67c0eb7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.855280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.856034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.856324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.856651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.857401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.862799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.896774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 849/1695] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:23.903215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:23.927173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab4f9-a255-4c4a-be47-a6d08111deb5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:23.947323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.948974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.949417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.949878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] PATH_INFO: `/v3/projects/06def8463fc840cdbd407353e1f48362/users/b720a53a8d2f4402b6ae3fffbc282182/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.953392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06def8463fc840cdbd407353e1f48362, user_id=b720a53a8d2f4402b6ae3fffbc282182, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.959942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a51c91-c01d-4edc-b3b8-eea7bfb1e163 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:23.977023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:23.977731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:23.981530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:23.981784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:23.982576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:23.997424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 850/1696] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:23 2022] PUT /identity/v3/projects/06def8463fc840cdbd407353e1f48362/users/b720a53a8d2f4402b6ae3fffbc282182/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:23.997592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:24.008050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4975-a351-44e4-bf6a-18e963e8a72f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.008642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4975-a351-44e4-bf6a-18e963e8a72f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.010236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4975-a351-44e4-bf6a-18e963e8a72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.101832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4f8f4975-a351-44e4-bf6a-18e963e8a72f None None] MFA Rules not processed for user `b720a53a8d2f4402b6ae3fffbc282182`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:24.106761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4f8f4975-a351-44e4-bf6a-18e963e8a72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:24.130603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 847/1697] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:23 2022] POST /identity/v3/users => generated 287 bytes in 713 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:24.131604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.167625 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dd2d9-605e-4e00-b02a-ad0c3bbc2293 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:24.325544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:24.332926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:24.333194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.333477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.333855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.335310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:24.337012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:24.356500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 851/1698] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:24.358291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.358788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.359077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.367407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 848/1699] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:24.368742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.395018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91effc-9ec5-43ca-b3c3-039d9155bccd admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:24.399313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] MFA Rules not processed for user `23b63d44bb304e97a38acc6d0b791c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:24.400925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:24.412318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:24.413124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.413519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.413896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.414921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:24.417623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:24.447827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 849/1700] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:24.448869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.452226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:24.538140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 852/1701] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:24.552742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.576924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed3f83-f1d0-4f99-96cd-946dd8c0f132 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:24.585864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:24.587216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.590657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.591165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] PATH_INFO: `/v3/projects/d11a34d9d6824eca941763667839c78e/users/cc603224add247778c396d8ab4a8dcb1/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.611071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11a34d9d6824eca941763667839c78e, user_id=cc603224add247778c396d8ab4a8dcb1, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:24.613971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b170a-3baa-4e04-9035-2c6ecc327150 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:24.626985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93721-aa6a-4071-a77c-c0759604588e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:24.633497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:24.636389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.637664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.637981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.676382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:24.677849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:24.701480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 850/1702] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:24.702415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.717377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:24.736905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 853/1703] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:24 2022] PUT /identity/v3/projects/d11a34d9d6824eca941763667839c78e/users/cc603224add247778c396d8ab4a8dcb1/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:24.742914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:24.744160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:24.745002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:24.795660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:24.799910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:24.921968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde83963-6c5b-40c7-a621-090dab8fd528 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:24.941348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 854/1704] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:24.952657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:24.954464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:25.031329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c687a-8e6a-457f-978d-584f80f5803f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:25.036282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:25.038430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:25.038812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:25.039198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:25.052506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:25.054231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:25.066638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 855/1705] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:25.100940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:25.101629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:25.101911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:25.102184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:25.243110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:25.293733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:25.305347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4405-4664-4c97-bb2d-191a093e6409 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:25.337129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 851/1706] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:25.965136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:25.968099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:25.999220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca9d68-9fd8-44e6-bce3-cf2ae1358489 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:26.006938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:26.007916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.008201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.008478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] PATH_INFO: `/v3/users/50d01be572d24e5db36c2863ab15f11a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.010489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d01be572d24e5db36c2863ab15f11a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:26.012561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:26.192317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80f4e-c901-45ec-876d-ca48ef3fdaa8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.192816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80f4e-c901-45ec-876d-ca48ef3fdaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.192888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80f4e-c901-45ec-876d-ca48ef3fdaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.276846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-01f80f4e-c901-45ec-876d-ca48ef3fdaa8 None None] MFA Rules not processed for user `a2cb1079722947dfa37946c03b7297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:26.282292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-01f80f4e-c901-45ec-876d-ca48ef3fdaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:26.316697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d01be572d24e5db36c2863ab15f11a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:26.334048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d01be572d24e5db36c2863ab15f11a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:26.347712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d01be572d24e5db36c2863ab15f11a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:26.350120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d01be572d24e5db36c2863ab15f11a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:26.432917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 852/1707] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:26.444098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.444459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.444806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.475986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] MFA Rules not processed for user `a2cb1079722947dfa37946c03b7297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:26.479744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:26.480265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a5fa7ea3-97a4-4eb6-9704-4940f103a6a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d01be572d24e5db36c2863ab15f11a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:26.503583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 853/1708] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 21 10:46:26.513500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:26.515195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:26.525146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 856/1709] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:25 2022] DELETE /identity/v3/users/50d01be572d24e5db36c2863ab15f11a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:26.538932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.539460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.540110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.590845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f513f9b-7890-4d02-a245-2661b0908e0e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:26.621143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:26.623862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:26.777113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:26.777113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.777113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.777113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.820014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 857/1710] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:26.821195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:26.827953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:26.852513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a666-f91b-48ca-97c6-e2beca21cdab None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:26.869866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:26.871103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.871532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.871952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] PATH_INFO: `/v3/users/3c2ea0a1fbde4b0ca7d69269f173cb5e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.875211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2ea0a1fbde4b0ca7d69269f173cb5e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:26.877541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:26.894835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:26.897880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdda56-62d8-4576-9b29-ecf4ac11f72d service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:26.913904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 854/1711] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:26.916812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:26.917490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:26.918935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:26.929681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:26.967448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] MFA Rules not processed for user `ad7606b56fec4723b733f838ec34a17e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:26.969791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.094178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 855/1712] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:27.094178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:27.094178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.109422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:27.110271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c988-7b2c-4ce4-b06d-3e774ceb8b27 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:27.119507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:27.119507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:27.119507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:27.119996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] PATH_INFO: `/v3/users/a9c99f002ea84c7d8127b15e2f9a6406` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:27.123813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c99f002ea84c7d8127b15e2f9a6406)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:27.126427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:27.173769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2ea0a1fbde4b0ca7d69269f173cb5e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.184779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2ea0a1fbde4b0ca7d69269f173cb5e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.185541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2ea0a1fbde4b0ca7d69269f173cb5e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.186193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2ea0a1fbde4b0ca7d69269f173cb5e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.275846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2ea0a1fbde4b0ca7d69269f173cb5e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.332425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 858/1713] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:26 2022] DELETE /identity/v3/users/3c2ea0a1fbde4b0ca7d69269f173cb5e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:27.339645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:27.341604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.423500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c99f002ea84c7d8127b15e2f9a6406'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.457473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c99f002ea84c7d8127b15e2f9a6406'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.556964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1182f-442e-4d62-ae57-48118f1a8642 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:27.567236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:27.568065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:27.568408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:27.568753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:27.570746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.576294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c99f002ea84c7d8127b15e2f9a6406'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.577188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c99f002ea84c7d8127b15e2f9a6406'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.578129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e650619-1a4f-44d5-b0a8-b96dca504453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c99f002ea84c7d8127b15e2f9a6406'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:27.622255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 856/1714] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:27 2022] DELETE /identity/v3/users/a9c99f002ea84c7d8127b15e2f9a6406 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:27.626486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b03a-ab17-400e-ad8c-f8d7c96715b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:27.627918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b03a-ab17-400e-ad8c-f8d7c96715b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:27.628479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b03a-ab17-400e-ad8c-f8d7c96715b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:27.656035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ef85b03a-ab17-400e-ad8c-f8d7c96715b5 None None] MFA Rules not processed for user `ad7606b56fec4723b733f838ec34a17e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:27.659303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ef85b03a-ab17-400e-ad8c-f8d7c96715b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.722901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:27.724360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e60eb-3513-4050-82a5-9cbefbdb752b service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:27.737612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 859/1715] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:27.739039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:27.739486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:27.740054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:27.762660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:27.766357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.802379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 857/1716] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:27.804248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:27.804611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:27.805215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:27.849466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:27.859837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:27.909964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 860/1717] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:27.911307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:27.913878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.029363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 858/1718] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:28.029363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:28.035025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.099857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f5494-360c-4a1d-ab30-0b6304bea959 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.108815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.109484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.110043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.110043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.112510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.127875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.129631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc59e22-05a1-4e55-941c-e47a98a1bc42 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.133271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8d40-a69d-4220-b58d-c2a0889dbbf4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.140599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.141393 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.141685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.141948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.148058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 861/1719] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:28.149829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.150119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.150399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.168835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:28.173656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.175178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.210775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 859/1720] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:28.211670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:28.238887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50821f-5c37-4428-8a1b-30b1efdaa683 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.240141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] MFA Rules not processed for user `cc603224add247778c396d8ab4a8dcb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:28.241813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.251690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.252354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.252636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.253199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.279034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.280637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b3004-96a3-4992-9267-752b092c253f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.292457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:28.298229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 860/1721] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:28.300030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cdde-4433-43cd-b187-04e22cc9cb1a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.300319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cdde-4433-43cd-b187-04e22cc9cb1a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.300595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cdde-4433-43cd-b187-04e22cc9cb1a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.302556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 861/1722] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 21 10:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:28.309576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.309932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.310130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.402923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:28.404622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.418059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 862/1723] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:28.420287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:28.422769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.434301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 862/1724] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:28.459221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:28.460847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:28.474071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad592b-16e1-4268-945b-b87b3c6c3933 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.481879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ee623-0c4b-442d-ae46-67524f9a851b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.491141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.491832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.492116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.492397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.493106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.497234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.499142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.499813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.500098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.500375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.520410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.523829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.544416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 863/1725] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:28.632412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 863/1726] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:28 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:28.637661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:28.653712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d75fa-f587-48fd-9f70-ba59c8c12294 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:28.783419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:28.821024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:28.821876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:28.822161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:28.822634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:28.823394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:28.825367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:28.972642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 864/1727] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:28 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:28.973741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:29.011054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c693dd8-a314-4d90-a370-5815b3f282b8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:29.035368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:29.036413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.036884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.037328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.038249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:29.041389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:29.057781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 864/1728] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:29.061449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:29.075944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b735b8-95d0-4e62-98aa-ac91667f3b8a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:29.094102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:29.095133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.095427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.095785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] PATH_INFO: `/v3/projects/3c88d44c963e4ad4b4915bdcea315848/users/069843fb01d2404c931f062d6b2ec2b0/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.133151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c88d44c963e4ad4b4915bdcea315848, user_id=069843fb01d2404c931f062d6b2ec2b0, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:29.149201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:29.266258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 865/1729] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:29 2022] PUT /identity/v3/projects/3c88d44c963e4ad4b4915bdcea315848/users/069843fb01d2404c931f062d6b2ec2b0/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:29.267654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:29.280857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8678864-57e4-4ae3-a2e2-f51deab70c61 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:29.369607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:29.472083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:29.473014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.473566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.474082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.474520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:29.476390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cadb56-67b3-4513-8182-a46fc4359e93 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:29.494157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 865/1730] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:29.498199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:29.509010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673e98c-0728-4111-9891-c5c44aa70d6b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:29.524208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:29.525676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.525966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.526244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] PATH_INFO: `/v3/projects/3c88d44c963e4ad4b4915bdcea315848/users/069843fb01d2404c931f062d6b2ec2b0/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.531866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c88d44c963e4ad4b4915bdcea315848, user_id=069843fb01d2404c931f062d6b2ec2b0, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:29.534631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b510d7b-76bc-4b5d-b63c-61aad6814466 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:29.565543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 866/1731] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:29 2022] PUT /identity/v3/projects/3c88d44c963e4ad4b4915bdcea315848/users/069843fb01d2404c931f062d6b2ec2b0/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:29.724619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.724916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.725204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.734560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 866/1732] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 21 10:46:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:29.740743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:29.741037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:29.741444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:29.765105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:29.770250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:29.810360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:29.811644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:29.835964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:30.009189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 867/1733] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:46:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:30.015985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4a1e0-8426-4d85-8eea-020530af96d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:30.024502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:30.025382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:30.025696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:30.026213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] PATH_INFO: `/v3/projects/0710a10e83c142349cd324ef9a421746` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:30.033952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0710a10e83c142349cd324ef9a421746)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:30.037197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:30.063390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:30.065091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:30.143519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0d7c-2f24-48a4-8e1d-56a332e63ac6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:30.161148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:30.162042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:30.162328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:30.162630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] PATH_INFO: `/v3/projects/3503902c27b8436ea380375798e96d9a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:30.166919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3503902c27b8436ea380375798e96d9a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:30.168921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:30.309288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0710a10e83c142349cd324ef9a421746', 'request_id': 'req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.474922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0710a10e83c142349cd324ef9a421746', 'request_id': 'req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.489312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0710a10e83c142349cd324ef9a421746', 'request_id': 'req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.515587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3503902c27b8436ea380375798e96d9a', 'request_id': 'req-86032f8a-525f-4860-a147-a3ad394af8ee'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.521961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] The token cache is being invalidate because project 0710a10e83c142349cd324ef9a421746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:30.522461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.525627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d15f3222-ea9c-4de5-b21e-6060b7aadfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.528064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 867/1734] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:29 2022] DELETE /identity/v3/projects/0710a10e83c142349cd324ef9a421746 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:30.542139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:30.542740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:30.549506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:30.554291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 868/1735] 10.222.0.48 () {54 vars in 961 bytes} [Thu Jul 21 10:46:30 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:30.555598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:30.565990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:30.641105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3503902c27b8436ea380375798e96d9a', 'request_id': 'req-86032f8a-525f-4860-a147-a3ad394af8ee'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.656821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3503902c27b8436ea380375798e96d9a', 'request_id': 'req-86032f8a-525f-4860-a147-a3ad394af8ee'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.692690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] The token cache is being invalidate because project 3503902c27b8436ea380375798e96d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:30.693291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.694435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:30.697058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 868/1736] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:30 2022] DELETE /identity/v3/projects/3503902c27b8436ea380375798e96d9a => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:30.698567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:30.703720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:30.814633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89c522-5700-4094-9815-6e28fc9b0b30 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:30.840908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:30.842302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:30.844702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:30.846455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] PATH_INFO: `/v3/projects/8f934391f10a47069cbfd9bfe3573e02` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:30.848498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f934391f10a47069cbfd9bfe3573e02)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:30.850861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:30.869220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:30.928244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032f8a-525f-4860-a147-a3ad394af8ee admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:30.933741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:30.934740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:30.935325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:30.935784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:30.941017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:30.954118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:30.955865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:30.977225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 869/1737] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:30.981650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:30.999659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f934391f10a47069cbfd9bfe3573e02', 'request_id': 'req-83965a63-409d-4e2c-9da5-9bb371b8fd12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.006182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:31.031890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dadde-b367-4158-9900-4e757287135f service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:31.103957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f934391f10a47069cbfd9bfe3573e02', 'request_id': 'req-83965a63-409d-4e2c-9da5-9bb371b8fd12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.112112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f934391f10a47069cbfd9bfe3573e02', 'request_id': 'req-83965a63-409d-4e2c-9da5-9bb371b8fd12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.140700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] The token cache is being invalidate because project 8f934391f10a47069cbfd9bfe3573e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:31.147023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.147981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-83965a63-409d-4e2c-9da5-9bb371b8fd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.158889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 869/1738] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:30 2022] DELETE /identity/v3/projects/8f934391f10a47069cbfd9bfe3573e02 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:31.160658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:31.162200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:31.162200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:31.178701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:31.179847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:31.182029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:31.183172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] PATH_INFO: `/v3/users/47ea1d8871944c90995edab42df0ed27` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:31.185631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ea1d8871944c90995edab42df0ed27)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:31.195354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:31.219075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:31.222170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:31.345400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 870/1739] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Jul 21 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:31.347064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:31.349460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:31.367962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efa99-9b6d-4e0f-857d-012832e95705 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:31.379913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:31.381389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:31.381823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:31.382298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] PATH_INFO: `/v3/users/b720a53a8d2f4402b6ae3fffbc282182` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:31.384460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b720a53a8d2f4402b6ae3fffbc282182)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:31.386813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:31.597667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ea1d8871944c90995edab42df0ed27'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.614495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ea1d8871944c90995edab42df0ed27'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.727468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b720a53a8d2f4402b6ae3fffbc282182'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.729220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ea1d8871944c90995edab42df0ed27'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.729942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ea1d8871944c90995edab42df0ed27'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.730598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1daef8da-7dbd-4aa2-8fab-065136b758c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ea1d8871944c90995edab42df0ed27'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.743246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b720a53a8d2f4402b6ae3fffbc282182'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.746613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b720a53a8d2f4402b6ae3fffbc282182'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.748295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b720a53a8d2f4402b6ae3fffbc282182'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.793844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 870/1740] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:30 2022] DELETE /identity/v3/users/47ea1d8871944c90995edab42df0ed27 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:31.800207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:31.807348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:31.807632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:31.851234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f74f44c-9a13-4b8d-aea8-3bfa1f8a2378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b720a53a8d2f4402b6ae3fffbc282182'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:31.874621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] MFA Rules not processed for user `a2cb1079722947dfa37946c03b7297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:31.877281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:31.895921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 871/1741] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:31 2022] DELETE /identity/v3/users/b720a53a8d2f4402b6ae3fffbc282182 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:31.898956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3be7-cd22-4adb-9ded-2add72c672f5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:31.903757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3be7-cd22-4adb-9ded-2add72c672f5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:31.904345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3be7-cd22-4adb-9ded-2add72c672f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:31.964663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b05a3be7-cd22-4adb-9ded-2add72c672f5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:31.969780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b05a3be7-cd22-4adb-9ded-2add72c672f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.037864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 871/1742] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:32.039102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:32.041859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.204259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 872/1743] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:32.206343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de89e27-a836-4c55-bca1-fd3b072d44a2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.206343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de89e27-a836-4c55-bca1-fd3b072d44a2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.206343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de89e27-a836-4c55-bca1-fd3b072d44a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.238343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4de89e27-a836-4c55-bca1-fd3b072d44a2 None None] MFA Rules not processed for user `23b63d44bb304e97a38acc6d0b791c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:32.241283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4de89e27-a836-4c55-bca1-fd3b072d44a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.280181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303d31-ee1f-4a7a-aaa9-1b46055351ba None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:32.326329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 873/1744] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:32.334442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.334756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.335313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.362215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:32.364260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.375293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 874/1745] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:32.381037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:32.412155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effb651-88ab-4882-96e1-3f6374c7b3fe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:32.420768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:32.421397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.421737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.422028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.424820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.430090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:32.445381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:32.446140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.446719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.447290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.452487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:32.452822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:32.458377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:32.470049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:32.475781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536e27b-0efb-4752-bb34-805076e70574 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:32.501843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 875/1746] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:32.510239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:32.525981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:32.541878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103958b2-5c05-44a8-88db-1658c894ab55 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:32.553689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:32.557059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.559017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.560299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.563543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:32.565956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:32.583488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 872/1747] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:32.585355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.593241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.594049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.686852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] MFA Rules not processed for user `906e4bc4e9cc45ecb01cfca7918e461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:32.688158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:32.768963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 876/1748] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:32 2022] POST /identity/v3/projects => generated 375 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:32.773061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:32.856334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 873/1749] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:32.912534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6387f-0646-4d6f-8c2d-5d78557bd94c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:32.926038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:32.926831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:32.927123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:32.927408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:32.928130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:32.930117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.041789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 877/1750] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:32 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:33.046051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.079752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564564c-2ab1-4d42-bed4-4ee5ff093820 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.102087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.102087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.102087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.103349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.104272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.110888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.127836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.129836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:33.149087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:33.160899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dccb6-feb9-4c05-a18d-1e0a1a99ed3a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.177015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.177986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.178342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.178718 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] PATH_INFO: `/v3/projects/06def8463fc840cdbd407353e1f48362` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.180060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06def8463fc840cdbd407353e1f48362)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.183419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.208910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 874/1751] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:33.216658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.233261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1224a6-1bfb-4778-85cd-464a47d220de admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.241669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.241669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.241669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.241669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.242115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.243234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.260243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 875/1752] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:33.271627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.287095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a39e4e-cb78-43f4-91e3-16b234c6f2c3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.298527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.299708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.300161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.301220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] PATH_INFO: `/v3/projects/6047a462ca234458a329166d92855012/users/0643be0e26fb41b7adc76c759cea339d/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.355915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06def8463fc840cdbd407353e1f48362', 'request_id': 'req-34059903-649f-40d9-a116-fc9f125f8226'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:33.368561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6047a462ca234458a329166d92855012, user_id=0643be0e26fb41b7adc76c759cea339d, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.372136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f27a32-d470-49cb-ba43-cbebd861da0c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.502243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06def8463fc840cdbd407353e1f48362', 'request_id': 'req-34059903-649f-40d9-a116-fc9f125f8226'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:33.518070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 876/1753] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:33 2022] PUT /identity/v3/projects/6047a462ca234458a329166d92855012/users/0643be0e26fb41b7adc76c759cea339d/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:33.520236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06def8463fc840cdbd407353e1f48362', 'request_id': 'req-34059903-649f-40d9-a116-fc9f125f8226'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:33.535289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] The token cache is being invalidate because project 06def8463fc840cdbd407353e1f48362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:33.536031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:33.543583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-34059903-649f-40d9-a116-fc9f125f8226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:33.543583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 878/1754] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:33 2022] DELETE /identity/v3/projects/06def8463fc840cdbd407353e1f48362 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:33.543882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.543882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.543882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.547224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97178358-53c2-46b1-890c-74bac83c9ed5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.547919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97178358-53c2-46b1-890c-74bac83c9ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.548464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97178358-53c2-46b1-890c-74bac83c9ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.593667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-97178358-53c2-46b1-890c-74bac83c9ed5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:33.595147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-97178358-53c2-46b1-890c-74bac83c9ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:33.598730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] MFA Rules not processed for user `0643be0e26fb41b7adc76c759cea339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:33.600397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:33.755570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 879/1755] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:33.765161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.765530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.765885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.808015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:33.809795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:33.826403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 880/1756] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:33.835848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.847263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 877/1757] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:33.851266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.853634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef4bb3-ff8a-4064-8f4f-c7cc7518d14a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.857843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:33.859979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.860809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.861279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.861738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.862792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:33.863318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.865422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.892116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17dcb-83df-4e12-94ea-315de2ce3fa9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.900835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 881/1758] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:33.902070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:33.904016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.904702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.904983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.905255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.906842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.909673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:33.940741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d8554-be04-45d0-ab8b-e8f47ccb413b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:33.956642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:33.957443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:33.957837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:33.958193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:33.959104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:33.961161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.033634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 878/1759] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:33 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:34.038177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.051749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:34.148439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 882/1760] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:33 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:34.150771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.329096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54597850-251b-4a85-aac8-c25065a1419e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.334933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0892-8554-400c-8c15-169ca1f35a96 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.354961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.358110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.358412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.358854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] PATH_INFO: `/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.370348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.370348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.370348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.370348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.370815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dbb38a56eb4190b76fcfd487f4af43)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.373237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.377190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.379925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.505171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 883/1761] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:34 2022] POST /identity/v3/users => generated 294 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:34.506138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.526251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cfb26-bdf4-415c-9a84-8f962f7890ef admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.535354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.536155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.536504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.536874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.541007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.554685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.561348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dbb38a56eb4190b76fcfd487f4af43', 'request_id': 'req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:34.659189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dbb38a56eb4190b76fcfd487f4af43', 'request_id': 'req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:34.672152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dbb38a56eb4190b76fcfd487f4af43', 'request_id': 'req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:34.701938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] The token cache is being invalidate because project f8dbb38a56eb4190b76fcfd487f4af43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:34.702201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 884/1762] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:34 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:34.702714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:34.703001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.705099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:34.707612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 879/1763] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:34 2022] DELETE /identity/v3/projects/f8dbb38a56eb4190b76fcfd487f4af43 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:34.708821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.716302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:34.729652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ccd35-aef7-42b7-8caa-ebd5b5557767 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.817361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1636b6-9517-49ad-a798-d8ae63ea7bfc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.820198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.820198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.820742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.821138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.822931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.825412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251f266-9ca6-4d69-81ba-29ff2677c169 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.838133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:34.845103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 885/1764] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:34.847985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.848123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.848585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.848585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.849831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.851922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.853619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.855616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.862877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.879714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:34.880856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:34.886400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 880/1765] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:34.887374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.889158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:34.904305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6228-1cac-4f42-bbba-30cd89e92a82 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.928983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.929948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.930229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.930509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] PATH_INFO: `/v3/projects/24606728b9b24894abb5e7652e3054a3/users/710915bd7bac4d76adad8575e240fc0a/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.935416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 886/1766] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:34.938249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:34.961712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24606728b9b24894abb5e7652e3054a3, user_id=710915bd7bac4d76adad8575e240fc0a, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.963931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e599c8-1e7b-4b60-8657-4d8634bd9c45 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:34.965046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:34.976913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:34.979456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:34.979852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:34.980237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:34.981685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:34.983407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95c7fb-91d0-48a9-9cba-f7f83e37561b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:35.001877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 887/1767] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:35.003215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.003614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.003999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.035423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:35.036723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:35.049212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 888/1768] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:35.052840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:35.068712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c0d2-61cb-43cd-a590-757e150613ed None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:35.091727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 881/1769] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:34 2022] PUT /identity/v3/projects/24606728b9b24894abb5e7652e3054a3/users/710915bd7bac4d76adad8575e240fc0a/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:35.093129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:35.097640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:35.098577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.099088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.099285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] PATH_INFO: `/v3/projects/70e97b51f70e4edea306f725b1fde070/users/13ffc813a32b498687e8cf25a5403c1a/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.124860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e97b51f70e4edea306f725b1fde070, user_id=13ffc813a32b498687e8cf25a5403c1a, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:35.130003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:35.166271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:35.245607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 889/1770] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:35 2022] PUT /identity/v3/projects/70e97b51f70e4edea306f725b1fde070/users/13ffc813a32b498687e8cf25a5403c1a/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:35.245607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:35.260822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280fba7-1361-49bd-a689-32509e82c3a5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:35.397894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:35.401270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.401885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.402704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.404380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:35.404704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:35.406831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17af5e-1895-487f-b58d-369eea0f53d9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:35.434209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 882/1771] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:35.437972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.438708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.438999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.459400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533891f7-10a9-4c41-9132-8083222b83ba admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:35.465653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:35.468388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.469658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.470157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.471248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:35.473373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:35.507934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 890/1772] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:35.512519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:35.514691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] MFA Rules not processed for user `13ffc813a32b498687e8cf25a5403c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:35.516274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:35.534525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45c9e-249f-487e-a300-9a9f666034ee admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:35.542536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:35.544105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.544538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.545360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.546246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:35.549356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e5fec-72f8-4840-8924-91b51542304e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:35.720444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 883/1773] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:35.721705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:35.747440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 891/1774] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:35 2022] POST /identity/v3/projects => generated 389 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:35.748868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.749163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.749444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.782749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97bbb1-28ed-4400-aa16-10798f710e11 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:35.836635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] MFA Rules not processed for user `069843fb01d2404c931f062d6b2ec2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:35.838477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:35.950981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:35.952733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:35.954979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:35.955395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] PATH_INFO: `/v3/projects/24606728b9b24894abb5e7652e3054a3/users/710915bd7bac4d76adad8575e240fc0a/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:35.965406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24606728b9b24894abb5e7652e3054a3, user_id=710915bd7bac4d76adad8575e240fc0a, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:35.975209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:36.027203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 884/1775] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:35 2022] PUT /identity/v3/projects/24606728b9b24894abb5e7652e3054a3/users/710915bd7bac4d76adad8575e240fc0a/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:36.036191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:36.148345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 892/1776] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:36.149020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:36.151020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:36.463412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb83a1-4d65-4816-bf1d-79f342fc5fe3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:36.463412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:36.463412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.463412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.464518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.464518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:36.464518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc778-9737-4e32-a964-fa86431f945d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:36.465141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78ad59-96c4-4527-ad25-f730cfa19cea None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:36.465141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:36.465141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.465141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.465727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.465727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:36.475614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:36.476940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:36.487863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 893/1777] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:36.488316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:36.500035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631d00c-f6b7-4a95-9525-1642f969fb4b service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:36.507076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:36.509880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.509880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.509880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.509880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:36.511185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:36.553041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 885/1778] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:36 2022] POST /identity/v3/projects => generated 365 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:36.554418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.554731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.555009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.589523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] MFA Rules not processed for user `710915bd7bac4d76adad8575e240fc0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:36.590797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:36.651306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 894/1779] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:36 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:36.651306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:36.651306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:36.801770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41fca-0cda-45db-a5be-f843bf9b2d86 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:36.810685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 886/1780] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:36.820805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:36.897834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:36.897834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.897834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.897834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.898111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:36.977394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0546545-4f30-4e9f-856a-74776362ea2f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:36.994254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:36.995333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:36.995676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:36.995961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:36.997517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.003478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.061239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.062997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.074300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 895/1781] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:37.075224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.087261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def190-0b9f-4507-87fd-00e88bf99b31 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.092208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.093000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.093435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.093789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.094784 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.097480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.160842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 896/1782] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:37.161204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.175341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:37.176055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 887/1783] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:36 2022] POST /identity/v3/users => generated 289 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:37.177210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.208101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066aa3d-e03c-46db-b33a-e00d90e065e4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.226167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.226402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:37.226928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.227215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.227652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.228342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4586a-0970-4d64-9d1f-157d18794601 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.228455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.231509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.245148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.246147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.246147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.246147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.278884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 888/1784] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:37.279891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.280615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:37.328999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9e1b-1332-4831-b055-4a6b3d381ced admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.356939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.357641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.357931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.358210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.362310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.368025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.399063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 889/1785] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:37.400578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.431169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab01d2-721d-4eb8-9823-4a195923daee admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.439679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.460917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.461325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.461606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] PATH_INFO: `/v3/projects/e4b739d619eb4b248df1c2596942d023/users/19db631db7f0445bac7171314120f58a/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.520431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:37.537112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b739d619eb4b248df1c2596942d023, user_id=19db631db7f0445bac7171314120f58a, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.538881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c86d9-8a93-41ab-9599-ccb8334b9c82 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.550660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:37.552471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:37.608799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 897/1786] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:37.616207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:37.661304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 890/1787] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:37 2022] PUT /identity/v3/projects/e4b739d619eb4b248df1c2596942d023/users/19db631db7f0445bac7171314120f58a/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:37.674766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609142d1-3794-4f30-8134-25e82d3e667d service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:37.676694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.677862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.678349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:37.767220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] MFA Rules not processed for user `19db631db7f0445bac7171314120f58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:37.769057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:37.929408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:37.931626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:37.936765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:37.938937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] PATH_INFO: `/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc/users/70398d96c2cc48358277be14343544e7/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.027233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185aa1c5f816447e8d19cfb8c8ca9fbc, user_id=70398d96c2cc48358277be14343544e7, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.034989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 891/1788] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:38.037745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:38.038254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.061506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18193663-895e-4fb5-98e1-1a179cda6f10 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:38.080961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:38.083779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.083779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.083779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.083779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.096175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.201575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 898/1789] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:37 2022] PUT /identity/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc/users/70398d96c2cc48358277be14343544e7/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:38.205564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:38.269222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 892/1790] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:38 2022] POST /identity/v3/projects => generated 389 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:38.270048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:38.517108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c6a2-1f7f-4a83-b1ae-f145bd004403 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:38.548850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa05e-8f27-42fd-afff-a8b18998edb6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:38.562677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:38.563612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.564922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.565389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.568249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.573085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6acd61-642f-4d86-8703-87a282c7b553 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.709512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:38.710207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.710693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.711132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.712617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.719732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.735795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 893/1791] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:38 2022] POST /identity/v3/users => generated 301 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:38.740517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dfab8-4f55-4d07-b630-12d06fb8b585 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.743707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dfab8-4f55-4d07-b630-12d06fb8b585 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.754131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dfab8-4f55-4d07-b630-12d06fb8b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.754853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 899/1792] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:38.761204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:38.785479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ca42-6ca4-4851-b6fb-25451c115ae0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:38.798601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:38.799467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.799854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.800233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.801069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.802808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.842501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 900/1793] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:38.844619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:38.871416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdfaf-8db9-4838-b190-fed4a4dd2241 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:38.890088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:38.891939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:38.892292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:38.896908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] PATH_INFO: `/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc/users/70398d96c2cc48358277be14343544e7/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:38.897183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:38.897282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185aa1c5f816447e8d19cfb8c8ca9fbc, user_id=70398d96c2cc48358277be14343544e7, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:38.907370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f30dfab8-4f55-4d07-b630-12d06fb8b585 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:38.910148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f30dfab8-4f55-4d07-b630-12d06fb8b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:38.928305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:38.948399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 901/1794] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:38 2022] PUT /identity/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc/users/70398d96c2cc48358277be14343544e7/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:38.949744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:39.148458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8477e1-5f70-4553-9e2b-b1c1f1a73613 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:39.171728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:39.173326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:39.177836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:39.180409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] PATH_INFO: `/v3/projects/0012154b5e934aa893cd17017b0bd96a/users/91c26890d0c342c6990bde0294c18761/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:39.207409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 894/1795] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:46:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:39.211775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:39.212703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:39.214113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:39.255575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:39.277844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0012154b5e934aa893cd17017b0bd96a, user_id=91c26890d0c342c6990bde0294c18761, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:39.279801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:39.332992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] MFA Rules not processed for user `70398d96c2cc48358277be14343544e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:39.335865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:39.439623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 902/1796] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:38 2022] PUT /identity/v3/projects/0012154b5e934aa893cd17017b0bd96a/users/91c26890d0c342c6990bde0294c18761/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:39.443663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:39.482597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 895/1797] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:39.492573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:39.612015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19285b4f-a0c4-48fa-9f3d-525f13f1ce48 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:39.664875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48fc8-b962-435e-aa36-a288087ea5de admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:39.670614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:39.671865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:39.671865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:39.672078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:39.673586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:39.677971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:39.705453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 903/1798] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:39.710393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:39.726040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:39.726746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:39.727037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:39.728707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:39.750510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb571a-f4ce-4881-baa6-309cfc1897cb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:39.776967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:39.777903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:39.778191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:39.778846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] PATH_INFO: `/v3/projects/0012154b5e934aa893cd17017b0bd96a/users/91c26890d0c342c6990bde0294c18761/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:39.793682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0012154b5e934aa893cd17017b0bd96a, user_id=91c26890d0c342c6990bde0294c18761, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:39.800072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:39.845735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 904/1799] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:39 2022] PUT /identity/v3/projects/0012154b5e934aa893cd17017b0bd96a/users/91c26890d0c342c6990bde0294c18761/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:39.846977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:39.968486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:39.970276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:39.978912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f71b-709a-4c6e-83ba-67ef6ed83702 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:39.983081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:39.983801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:40.003531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:40.003531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:40.011030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 896/1800] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:40.012278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:40.021585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:40.128095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:40.129608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bf2f4-beaf-4f97-9e18-e5d3e9cbedfa service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:40.146317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 905/1801] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:40.147955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:40.148344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:40.148630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:40.233046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] MFA Rules not processed for user `91c26890d0c342c6990bde0294c18761`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:40.242633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:40.424261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798385e-8fb5-4b47-9b8d-e6c77cae1cb4 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:40.477758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 906/1802] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:40.534192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:40.549402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17bd36-1f93-419a-b791-22ac786e85b3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:40.553398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:40.554073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:40.555216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:40.556146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:40.568405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:40.569985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:40.569985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:40.569985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:40.570318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:40.570793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:40.583745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 907/1803] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:40.590067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:40.657187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:40.662015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556a0ec-7679-4055-9599-1abcee01897d service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:40.670102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 897/1804] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:46:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:46:40.702050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:40.719311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356407-101c-4568-a4e1-0c5abf6a9f96 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:40.727191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:40.728484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:40.728873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:40.729370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:40.877058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:40.880388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:40.910525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 908/1805] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:40.943642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=1,keystone:SELECT=92,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:41.435556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:41.436136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:41.436752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:41.500023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:41.500984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:41.741969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 898/1806] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 21 10:46:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:41.789626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:41.935524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:41.936982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:41.982867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28416-86a9-4daf-998c-dad4a92d00f8 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:42.000090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:42.005388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:42.005661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:42.005945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:42.047987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:42.050166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1542d2a-bb8b-4676-864d-e4783bcf484f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:42.071295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 909/1807] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:42.164740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:42.289219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712c05-4903-481a-b06a-5e36ee39dd9a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:42.296717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:42.299794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:42.299794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:42.299794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] PATH_INFO: `/v3/users/19db631db7f0445bac7171314120f58a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:42.299794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19db631db7f0445bac7171314120f58a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:42.302016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:42.729542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19db631db7f0445bac7171314120f58a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:42.742316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19db631db7f0445bac7171314120f58a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:42.778292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:42.846090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19db631db7f0445bac7171314120f58a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:42.847532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19db631db7f0445bac7171314120f58a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:42.848508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19db631db7f0445bac7171314120f58a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:42.881638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 899/1808] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:42 2022] DELETE /identity/v3/users/19db631db7f0445bac7171314120f58a => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:42.886346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a063a-8484-4410-8f8d-9c29a6f23d34 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:42.886972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a063a-8484-4410-8f8d-9c29a6f23d34 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:42.887805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a063a-8484-4410-8f8d-9c29a6f23d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:42.916126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4b1a063a-8484-4410-8f8d-9c29a6f23d34 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:42.917842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4b1a063a-8484-4410-8f8d-9c29a6f23d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:43.027646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 910/1809] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:43.051116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:43.053834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:43.091193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:43.153086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64601a0-7539-42a7-aba6-3026cc03ca62 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:43.159184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:43.159184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:43.159184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:43.159184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:43.181276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:43.187874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aa756-e5fd-4df5-aed1-6a1690f76df8 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:43.200466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 900/1810] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:43.482043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00414ae-42a0-4081-969d-6bfa8ebc8e4a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:43.482336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00414ae-42a0-4081-969d-6bfa8ebc8e4a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:43.482649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00414ae-42a0-4081-969d-6bfa8ebc8e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:43.502886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c00414ae-42a0-4081-969d-6bfa8ebc8e4a None None] MFA Rules not processed for user `906e4bc4e9cc45ecb01cfca7918e461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:43.504427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c00414ae-42a0-4081-969d-6bfa8ebc8e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:43.590669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 911/1811] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:43.866432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:43.869049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:43.869517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:43.872306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 901/1812] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:46:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:43.894809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:43.896428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:43.896428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:43.931132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:43.931132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:43.945279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:43.948585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:43.954314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:43.989621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552f1a1-2edc-4032-9315-47475bfb79d1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:44.008279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:44.011328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:44.011883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:44.012378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] PATH_INFO: `/v3/projects/e4b739d619eb4b248df1c2596942d023` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:44.015441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b739d619eb4b248df1c2596942d023)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:44.017623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:44.035944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 912/1813] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:44.068127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:44.165054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b739d619eb4b248df1c2596942d023', 'request_id': 'req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.216398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b739d619eb4b248df1c2596942d023', 'request_id': 'req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.227337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b739d619eb4b248df1c2596942d023', 'request_id': 'req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.230882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:44.241478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] The token cache is being invalidate because project e4b739d619eb4b248df1c2596942d023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:44.241774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.242506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.247480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 902/1814] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:43 2022] DELETE /identity/v3/projects/e4b739d619eb4b248df1c2596942d023 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:44.248811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:44.260492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5d07e-e14e-40b4-963a-37bce2e81fbb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:44.342125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:44.343171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:44.343459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:44.343738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] PATH_INFO: `/v3/users/91c26890d0c342c6990bde0294c18761` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:44.345000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c26890d0c342c6990bde0294c18761)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:44.347007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:44.428950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:44.430781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:44.541515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414ad2-e51c-4cb8-b9b3-1867e2b8e858 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:44.549529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:44.550292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:44.550673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:44.551331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:44.553767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:44.589385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:44.598259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:44.607443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c26890d0c342c6990bde0294c18761'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.622019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 903/1815] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:44.630712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c26890d0c342c6990bde0294c18761'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.633430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c26890d0c342c6990bde0294c18761'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.637572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c26890d0c342c6990bde0294c18761'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.742885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c26890d0c342c6990bde0294c18761'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:44.791781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 913/1816] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:44 2022] DELETE /identity/v3/users/91c26890d0c342c6990bde0294c18761 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:44.946731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:44.949872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:44.973309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:45.041151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611283-21ef-44cd-a010-408a8cc78652 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:45.168942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:45.211436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:45.212832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:45.213357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:45.213739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:45.217881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:45.401543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:45.403126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef3be-4320-49bc-9958-6852fc25e31a service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:45.425261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 904/1817] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:45.439275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:45.689772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:45.902866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:46.958045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:46.960214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:47.193388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd585-9f32-4d11-a16b-5c4293eaaba0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:47.206915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:47.210842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.211687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.213677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] PATH_INFO: `/v3/projects/0012154b5e934aa893cd17017b0bd96a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.220043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0012154b5e934aa893cd17017b0bd96a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:47.223724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:47.438000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0012154b5e934aa893cd17017b0bd96a', 'request_id': 'req-e92991c7-fd08-4a55-8130-905f1a8e23cb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:47.542024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0012154b5e934aa893cd17017b0bd96a', 'request_id': 'req-e92991c7-fd08-4a55-8130-905f1a8e23cb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:47.550970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0012154b5e934aa893cd17017b0bd96a', 'request_id': 'req-e92991c7-fd08-4a55-8130-905f1a8e23cb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:47.554489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6786-03ea-47b1-b5a1-79af78805989 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.555010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6786-03ea-47b1-b5a1-79af78805989 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.555104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6786-03ea-47b1-b5a1-79af78805989 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.556645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 905/1818] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:46:47.580350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] The token cache is being invalidate because project 0012154b5e934aa893cd17017b0bd96a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:47.580654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:47.590053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e92991c7-fd08-4a55-8130-905f1a8e23cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:47.593541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 914/1819] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:46 2022] DELETE /identity/v3/projects/0012154b5e934aa893cd17017b0bd96a => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:47.598510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667711-5320-49b0-8826-6dea7656ca14 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.598816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667711-5320-49b0-8826-6dea7656ca14 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.599257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667711-5320-49b0-8826-6dea7656ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.621077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.621389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.621719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.635340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a0667711-5320-49b0-8826-6dea7656ca14 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:47.636584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a0667711-5320-49b0-8826-6dea7656ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:47.643721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:47.697434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:47.710233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:47.793136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 906/1820] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:47.803186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.803757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.806273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.860215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:47.861458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:47.880156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 915/1821] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:47.883867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 907/1822] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:47.888201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:47.909277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e81255-45c0-41b4-a12a-1235df7341a7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:47.916576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:47.917413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:47.917842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:47.918187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:47.919136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:47.919510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:47.921672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:47.955886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:47.958100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:47.958843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 916/1823] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:47.975857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:47.992411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902f2c5-623b-4f73-87ec-e75e9e85bd8f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:47.998227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7856d62-67fa-459f-a2c5-247f73210ab3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.003876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.004810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.005247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.006658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.007527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.011193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.011335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.012014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.012298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.012574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.053792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.056559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.101726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 908/1824] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:46:48.108616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:48.208877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 917/1825] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:47 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:48.215263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:48.231722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895bf7-1069-4b9f-802c-55e3bdc74f0c service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.371237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.372220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.375160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.375160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.375160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.376285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.394225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:48.480586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:48.482913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 909/1826] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:48 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:48.499845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3dbee-2fb3-40f5-8b7a-fba380f5624b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.513332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.514057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.514351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.514641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.515418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.517055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.545997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:48.548108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 918/1827] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:48.559344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecfd61-48e2-41a1-a3b2-aa1499222820 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.570289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.571214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.571501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.571881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.573120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.577940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650f218-9f66-41fc-809c-6859f6da823d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.591742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 910/1828] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:46:48.595900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:48.608317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aed71-d5a3-4793-af4f-fec6558e842c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.612908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.613830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.614122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.614350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] PATH_INFO: `/v3/projects/6ccaca39663849e683817d04510a1831/users/81f2c22c6f714debb169f951c551e023/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.629915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccaca39663849e683817d04510a1831, user_id=81f2c22c6f714debb169f951c551e023, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.632754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:48.716134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 919/1829] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:48 2022] PUT /identity/v3/projects/6ccaca39663849e683817d04510a1831/users/81f2c22c6f714debb169f951c551e023/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:48.728871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.729293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.729676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.761106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] MFA Rules not processed for user `81f2c22c6f714debb169f951c551e023`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:48.762574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:48.853408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 911/1830] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:48.857897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:48.885661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:48.947749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b803d7-257e-437e-b2ee-4f2443e0f0c0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:48.952827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:48.953510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:48.953798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:48.954081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:48.954834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:48.959246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.128193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 920/1831] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:48 2022] POST /identity/v3/projects => generated 387 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:49.131064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.142639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57feb314-cdb3-4b86-a635-56e5c2c51c26 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.202193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.202911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.203194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.203469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.204172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.206090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.336236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 912/1832] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:49 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:49.340675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.351460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004084bb-0429-4893-b2ed-c248dce881ac admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.356734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.357438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.357728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.358008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.358771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.360368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.372166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 921/1833] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:46:49.376314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.386891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6997-bd7d-4b54-af03-ca012f787733 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.391484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.392537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.392822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.393099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] PATH_INFO: `/v3/projects/86b342c0b091421ab7e23e80d59f6d90/users/81037c16739148f390afcf08814e08d8/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.407808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b342c0b091421ab7e23e80d59f6d90, user_id=81037c16739148f390afcf08814e08d8, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.410493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.480476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 913/1834] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:49 2022] PUT /identity/v3/projects/86b342c0b091421ab7e23e80d59f6d90/users/81037c16739148f390afcf08814e08d8/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:49.484576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.495941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749188c-f661-401a-85d7-27faa760c976 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.568827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.569573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.569920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.570271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.571117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.573402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71c861-cc09-4743-ad8f-d84a2d5dc833 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.584405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 922/1835] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:49.589018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.600506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664e91-56be-45fd-a2ee-ba549d5db5a4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.606872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.608019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.608444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.609129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] PATH_INFO: `/v3/projects/86b342c0b091421ab7e23e80d59f6d90/users/81037c16739148f390afcf08814e08d8/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.611412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b342c0b091421ab7e23e80d59f6d90, user_id=81037c16739148f390afcf08814e08d8, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.614444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.628550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 914/1836] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:49 2022] PUT /identity/v3/projects/86b342c0b091421ab7e23e80d59f6d90/users/81037c16739148f390afcf08814e08d8/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:49.638918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.639211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.639501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.688405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] MFA Rules not processed for user `81037c16739148f390afcf08814e08d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:49.689641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:49.758887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 923/1837] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:49.769980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:49.771803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:49.787952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:49.838496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defabe6e-b520-4ad0-8080-20cabb1d98e7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:49.838496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:49.838724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:49.838983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:49.839259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:49.854025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:49.854025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243ca3d-3466-4d71-907d-374ac524e1b8 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:49.861503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 915/1838] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:50.240242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:50.353987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:51.560250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:51.692434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a2ef8-6bb5-4a4f-b7e4-75f2d3ec4c23 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:51.697585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:51.698462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:51.698771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:51.699048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] PATH_INFO: `/v3/users/81f2c22c6f714debb169f951c551e023` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:51.700290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f2c22c6f714debb169f951c551e023)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:51.702240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:51.708193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:51.928430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f2c22c6f714debb169f951c551e023'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:51.947843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f2c22c6f714debb169f951c551e023'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:51.950028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f2c22c6f714debb169f951c551e023'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:51.951458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f2c22c6f714debb169f951c551e023'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:52.052554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f2c22c6f714debb169f951c551e023'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:52.083674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 924/1839] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:51 2022] DELETE /identity/v3/users/81f2c22c6f714debb169f951c551e023 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:52.089675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:52.090224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:52.090224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:52.108824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:52.110040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:52.190976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 916/1840] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:52.204475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:52.206361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:52.280538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd73b-9d64-4b3f-9c97-f6d80ab159e6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:52.284384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:52.285059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:52.285346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:52.285629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:52.295923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:52.297390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:52.307918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 925/1841] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:52.929340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:52.930737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:52.949700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7729fc-43d4-4c8f-a65f-dca2383f4452 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:52.954755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:52.955666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:52.955935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:52.956211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] PATH_INFO: `/v3/projects/6ccaca39663849e683817d04510a1831` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:52.957894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccaca39663849e683817d04510a1831)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:52.960273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:53.128924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccaca39663849e683817d04510a1831', 'request_id': 'req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.203302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccaca39663849e683817d04510a1831', 'request_id': 'req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.210027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccaca39663849e683817d04510a1831', 'request_id': 'req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.223117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] The token cache is being invalidate because project 6ccaca39663849e683817d04510a1831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:53.223416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.224092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.225685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 917/1842] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:52 2022] DELETE /identity/v3/projects/6ccaca39663849e683817d04510a1831 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:53.230467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:53.240472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca30e13-0dae-4f8a-b173-850df65aa384 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:53.318160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:53.319167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:53.319554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:53.319942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] PATH_INFO: `/v3/users/81037c16739148f390afcf08814e08d8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:53.321492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81037c16739148f390afcf08814e08d8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:53.323563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:53.549612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:53.564916 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81037c16739148f390afcf08814e08d8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.572559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81037c16739148f390afcf08814e08d8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.573213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81037c16739148f390afcf08814e08d8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.573837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81037c16739148f390afcf08814e08d8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.651838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd0bddab-9c7c-42df-8a9d-750dce849c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81037c16739148f390afcf08814e08d8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:53.693528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 926/1843] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:53 2022] DELETE /identity/v3/users/81037c16739148f390afcf08814e08d8 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:54.411359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:54.413621 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:54.434030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3808ce-cd41-4a8c-9d42-dff09fac1a95 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:54.508021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:54.518975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:54.519997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:54.520343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:54.520620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] PATH_INFO: `/v3/projects/86b342c0b091421ab7e23e80d59f6d90` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:54.521854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b342c0b091421ab7e23e80d59f6d90)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:54.527826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:54.662745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b342c0b091421ab7e23e80d59f6d90', 'request_id': 'req-d7e24abe-892f-4561-acbe-813477aaa042'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:54.746592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b342c0b091421ab7e23e80d59f6d90', 'request_id': 'req-d7e24abe-892f-4561-acbe-813477aaa042'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:54.755276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b342c0b091421ab7e23e80d59f6d90', 'request_id': 'req-d7e24abe-892f-4561-acbe-813477aaa042'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:54.777645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] The token cache is being invalidate because project 86b342c0b091421ab7e23e80d59f6d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:54.777948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:54.778769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d7e24abe-892f-4561-acbe-813477aaa042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:54.785569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 918/1844] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:54 2022] DELETE /identity/v3/projects/86b342c0b091421ab7e23e80d59f6d90 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:54.791841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:54.792385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:54.792385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:54.815842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:54.817237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:54.929314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 927/1845] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:54.937797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:54.938319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:54.939429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:54.993223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:54.994942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:55.020448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 919/1846] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:55.024989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.035856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab163c-d43b-4df0-a424-1f84b1663016 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.042202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.043404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.043847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.044278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.045130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:55.045630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.047397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:55.073848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 928/1847] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:55.078154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.090467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b8868-5a0b-4729-aa87-ed5c3bc2cb05 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.106835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.107755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.112162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.112162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.112162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.121392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:55.288341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 920/1848] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:55 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:46:55.289363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.313241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c17f0-a773-4648-b38a-db79cfe9cdd0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.408852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.409669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.410014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.410351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.411260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.415292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:55.528413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 929/1849] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:55 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:55.530517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.551108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fcaf-1365-4e50-aad5-15b01e976f72 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.561499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.563686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.565279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.565279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.565279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.567051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:55.586613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 921/1850] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:55.591680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.623328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411771ee-6b72-43ee-9db3-daae296bc609 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.628135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.628818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.629104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.629385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.630114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.635590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735148-752c-4ac4-b117-a1a1917bd82f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:55.652476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 930/1851] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:55.662021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:55.683766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6962dd-cf31-4556-9921-89a281208897 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:55.690726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:55.693988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:55.694517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:55.695131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] PATH_INFO: `/v3/projects/181c8fe009844fa2871fcdea66891ee8/users/51a2433e65f24324b5aa6a59d744cf43/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:55.716357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=181c8fe009844fa2871fcdea66891ee8, user_id=51a2433e65f24324b5aa6a59d744cf43, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:55.719097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:56.008455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 922/1852] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:55 2022] PUT /identity/v3/projects/181c8fe009844fa2871fcdea66891ee8/users/51a2433e65f24324b5aa6a59d744cf43/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:56.022025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:56.022581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:56.022683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:56.060513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:56.064610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] MFA Rules not processed for user `51a2433e65f24324b5aa6a59d744cf43`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:56.065951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:56.142943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 931/1853] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:56.155633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:56.157438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:56.241751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb85d0-d360-4f09-8421-04066918e862 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:56.245972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:56.247347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:56.247633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:56.247918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:56.266336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:56.268432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f606fb0-7ce9-4556-920d-0f76ed258e9d service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:56.283160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 923/1854] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:56.303310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:56.432540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f09ab2-09a4-4816-83bb-a5dbd015cabe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:56.445188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:56.446781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:56.447297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:56.447585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] PATH_INFO: `/v3/users/51a2433e65f24324b5aa6a59d744cf43` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:56.448908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a2433e65f24324b5aa6a59d744cf43)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:56.450884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:56.730804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a2433e65f24324b5aa6a59d744cf43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:56.737752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a2433e65f24324b5aa6a59d744cf43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:56.738426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a2433e65f24324b5aa6a59d744cf43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:56.739069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a2433e65f24324b5aa6a59d744cf43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:56.821177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a2433e65f24324b5aa6a59d744cf43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:56.840184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 932/1855] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:46:56 2022] DELETE /identity/v3/users/51a2433e65f24324b5aa6a59d744cf43 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:56.844548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:56.845046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:56.845163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:56.862981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:56.864177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:56.880929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:56.926365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 924/1856] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:56.937102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:56.938576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:56.999143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afec2c8-a32f-45db-8f10-2aa40818e9e3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:57.002776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:57.008112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.008112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.008112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.015878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:57.017452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc9e5e-0b8a-4336-8783-87191577689f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:57.028188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 933/1857] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:46:57.540878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:57.542748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:57.553738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:57.556029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed4d5-f486-42e5-88c7-98ea26fffbf7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:57.561447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:57.562620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.563061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.563347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] PATH_INFO: `/v3/projects/181c8fe009844fa2871fcdea66891ee8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.564590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=181c8fe009844fa2871fcdea66891ee8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:57.566579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:57.674987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181c8fe009844fa2871fcdea66891ee8', 'request_id': 'req-2885eba3-e08e-402e-8771-83441ccfd1dd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:57.759579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181c8fe009844fa2871fcdea66891ee8', 'request_id': 'req-2885eba3-e08e-402e-8771-83441ccfd1dd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:57.772745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181c8fe009844fa2871fcdea66891ee8', 'request_id': 'req-2885eba3-e08e-402e-8771-83441ccfd1dd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:57.792768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] The token cache is being invalidate because project 181c8fe009844fa2871fcdea66891ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:46:57.803573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:57.803573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2885eba3-e08e-402e-8771-83441ccfd1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:46:57.803573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 925/1858] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:46:57 2022] DELETE /identity/v3/projects/181c8fe009844fa2871fcdea66891ee8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:57.803836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc102e-e103-4358-8908-27c82406199a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.803836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc102e-e103-4358-8908-27c82406199a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.803836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc102e-e103-4358-8908-27c82406199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.819496 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4bfc102e-e103-4358-8908-27c82406199a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:57.822727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4bfc102e-e103-4358-8908-27c82406199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:57.894474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 934/1859] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:46:57.902406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.906830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.906830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.923131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:57.924407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:57.933984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 926/1860] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:57.938465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc102e-e103-4358-8908-27c82406199a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:57.949572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc102e-e103-4358-8908-27c82406199a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:57.954515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:57.955288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.955569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.955875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.956601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:46:57.956904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:57.958590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:57.971501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 935/1861] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:46:57.976424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:57.987142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154eda1-4903-43a7-a4b8-88dae60659e4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:57.991769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:57.997397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:57.997397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:57.997397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:57.997397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:57.997397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.097568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 927/1862] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:57 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:58.098746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.110811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087ddfe-dfad-49e5-8309-c7daed1fdb17 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.178106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.178910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.179080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.179363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.180993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.186845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.267866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 936/1863] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:58 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:58.272147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.283917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b732b6-ee5c-4548-a2bf-0cc14b4fa2db admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.289207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.290531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.290964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.291251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.291991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.294188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.307667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 928/1864] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:58.311991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.323014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cda84-30a9-496e-84ca-94df96333422 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.328285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.329044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.329488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.329788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.330506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.332148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebfa9f-d29a-454a-bb37-494589a9706a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.345163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 937/1865] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:58.351080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.362002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e089fd3-3c3c-41ae-885a-14e112caa930 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.367077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.368269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.368768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.369043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] PATH_INFO: `/v3/projects/dc0e0891981748c3b34c68c65a3bc781/users/80f3aba426f44deca103331aa7f37019/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.392327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0e0891981748c3b34c68c65a3bc781, user_id=80f3aba426f44deca103331aa7f37019, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.395057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.462668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 929/1866] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:58 2022] PUT /identity/v3/projects/dc0e0891981748c3b34c68c65a3bc781/users/80f3aba426f44deca103331aa7f37019/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:46:58.473400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.473690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.473971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.502789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] MFA Rules not processed for user `80f3aba426f44deca103331aa7f37019`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:58.504010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:58.523479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:58.573743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 938/1867] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:58.578593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.628758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:58.647816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca81c-efa6-4fa5-baeb-13e67fe2ffee admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.653602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.654276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.654584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.654866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.655565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.657486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.760878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 930/1868] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:58 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:58.764902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.776677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cd991-62e7-4211-97f2-c7e69603d416 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:58.839987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:58.840673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:58.840955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:58.841229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:58.841935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:58.843924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:58.973771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 939/1869] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:58 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:58.974793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:58.995455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401220-8c87-4e97-8dd9-4c66a0b59b09 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.002739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.004464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.004930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.005702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.007009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.009846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.030296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 931/1870] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:46:59.032152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.042697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e094e8f-b7ef-4dcf-9798-52dd26f08d2e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.049974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.050922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.051303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.051706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.055366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.060466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fde464-0675-4aad-a68d-e9f8c1efa054 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.068290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 940/1871] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:59.073210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.084720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9819b2-db85-4307-b2da-5e54f9e2699e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.090578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.091487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.091773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.092069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] PATH_INFO: `/v3/projects/b1994f9771eb4baab928b816c989f496/users/a8c639539c764964ac0d718361c299e5/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.109184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1994f9771eb4baab928b816c989f496, user_id=a8c639539c764964ac0d718361c299e5, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.111896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.181864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 932/1872] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:59 2022] PUT /identity/v3/projects/b1994f9771eb4baab928b816c989f496/users/a8c639539c764964ac0d718361c299e5/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:59.193794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.194324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.194450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.222817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] MFA Rules not processed for user `a8c639539c764964ac0d718361c299e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:46:59.224044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:46:59.291809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 941/1873] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:46:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:46:59.296282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.367553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0922338-bbb9-4674-aadc-94f23be7c9ef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.372533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.373203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.373489 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.373767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.374465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.376486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.483597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 933/1874] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:46:59 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:59.484791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.496189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd890bde-38f7-4317-bfeb-36689571dfc9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.524241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:46:59.555774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.556566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.556847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.557117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.557799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.559752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.684009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 942/1875] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:46:59 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:46:59.685799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.700638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d91217-424c-4562-9dd7-df609049e027 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.707306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.708129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.708531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.708904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.709728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.712630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.739169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 934/1876] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:59.744489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.755223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe82d-0895-49d7-9535-d8ec027666e2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.762562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.763674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.763959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.764869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] PATH_INFO: `/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a/users/28981d5e278e43388db1ef9c250802bd/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.783007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eb6ef2612c4adf9edc10c07ff5545a, user_id=28981d5e278e43388db1ef9c250802bd, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.785965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.878157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 943/1877] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:59 2022] PUT /identity/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a/users/28981d5e278e43388db1ef9c250802bd/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:46:59.881763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.892802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebd1e-05e4-4eea-bec6-5a6af360d46f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.957869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.958583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.958842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.959119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.959836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:46:59.961828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c77e-0f39-45b9-b6fd-56e98b03dfcf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:46:59.972906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 935/1878] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:46:59.978181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:46:59.988591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637f151-3575-4811-a190-865cc46b3d6a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:46:59.993889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:46:59.994788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:46:59.995072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:46:59.995347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] PATH_INFO: `/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a/users/28981d5e278e43388db1ef9c250802bd/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:46:59.997376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eb6ef2612c4adf9edc10c07ff5545a, user_id=28981d5e278e43388db1ef9c250802bd, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:00.000065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:00.010232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 944/1879] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:46:59 2022] PUT /identity/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a/users/28981d5e278e43388db1ef9c250802bd/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:00.020422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:00.020973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:00.020973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:00.027249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:00.049138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] MFA Rules not processed for user `28981d5e278e43388db1ef9c250802bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:00.050283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:00.140460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 936/1880] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:00.154365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:00.156223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:00.234475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84fe9b-ce20-44b7-9b03-ca5cf220168c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:00.240201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:00.241250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:00.241626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:00.242026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:00.314654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:00.316039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af30bc5-d14a-4cd0-9463-a93953d97787 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:00.327663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 945/1881] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:02.415421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:02.418297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:02.521282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f19450-9a9a-42aa-98d0-2f9f9860458c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:02.532536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:02.539239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:02.545133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:02.545133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] PATH_INFO: `/v3/users/13ffc813a32b498687e8cf25a5403c1a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:02.545133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ffc813a32b498687e8cf25a5403c1a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:02.547727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:02.815079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ffc813a32b498687e8cf25a5403c1a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:02.823858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ffc813a32b498687e8cf25a5403c1a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:03.102987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ffc813a32b498687e8cf25a5403c1a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:03.105331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ffc813a32b498687e8cf25a5403c1a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:03.105331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ffc813a32b498687e8cf25a5403c1a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:03.126956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 937/1882] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:02 2022] DELETE /identity/v3/users/13ffc813a32b498687e8cf25a5403c1a => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:03.137704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:03.138186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:03.138281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:03.159965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:03.161186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:03.195039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:03.225231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 946/1883] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:03.235653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:03.237426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:03.248378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:03.301746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7449c-f275-4117-90e3-5feed871d76e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:03.306010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:03.306704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:03.306989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:03.307265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:03.317388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:03.318878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:03.330214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 938/1884] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:03.824085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:03.825584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:03.868379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:03.870118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:03.892898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c8cb-c55e-4865-8872-7a539d521487 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:03.896687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403b160-2369-4b13-8ca8-5a8d9afc1ab3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:03.903803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:03.904458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:03.904795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:03.905076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:03.906524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:03.907781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:03.907909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:03.908204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:03.908547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] PATH_INFO: `/v3/projects/70e97b51f70e4edea306f725b1fde070` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:03.909817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e97b51f70e4edea306f725b1fde070)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:03.912218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:03.969641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:03.971131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:03.981283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 947/1885] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:04.078459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e97b51f70e4edea306f725b1fde070', 'request_id': 'req-7b154cf9-493b-4239-b585-8a65246e4e6b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.169426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e97b51f70e4edea306f725b1fde070', 'request_id': 'req-7b154cf9-493b-4239-b585-8a65246e4e6b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.176174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e97b51f70e4edea306f725b1fde070', 'request_id': 'req-7b154cf9-493b-4239-b585-8a65246e4e6b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.196673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] The token cache is being invalidate because project 70e97b51f70e4edea306f725b1fde070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:04.197075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.197714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.199671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 939/1886] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:03 2022] DELETE /identity/v3/projects/70e97b51f70e4edea306f725b1fde070 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:04.203362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:04.216356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dda657-baa6-4952-b0ee-4df2ff8aec26 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:04.284915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:04.285799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:04.286083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:04.286357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] PATH_INFO: `/v3/users/70398d96c2cc48358277be14343544e7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:04.287635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70398d96c2cc48358277be14343544e7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:04.289841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:04.491581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:04.503825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70398d96c2cc48358277be14343544e7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.511999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70398d96c2cc48358277be14343544e7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.512866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70398d96c2cc48358277be14343544e7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.513510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70398d96c2cc48358277be14343544e7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.627173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5f5ef20f-50b5-4484-9359-6de8e9749bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70398d96c2cc48358277be14343544e7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:04.653240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 948/1887] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:04 2022] DELETE /identity/v3/users/70398d96c2cc48358277be14343544e7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:05.085137 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.087952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:05.103944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b154cf9-493b-4239-b585-8a65246e4e6b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.163814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.164828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.165049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.165311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] PATH_INFO: `/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.166517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=185aa1c5f816447e8d19cfb8c8ca9fbc)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.168469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.300217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:05.314051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185aa1c5f816447e8d19cfb8c8ca9fbc', 'request_id': 'req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:05.375738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185aa1c5f816447e8d19cfb8c8ca9fbc', 'request_id': 'req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:05.382143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185aa1c5f816447e8d19cfb8c8ca9fbc', 'request_id': 'req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:05.402425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] The token cache is being invalidate because project 185aa1c5f816447e8d19cfb8c8ca9fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:05.402748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:05.403448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fd6cd4e8-f278-40e7-92a7-59f81dc5c84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:05.408649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 940/1888] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:05 2022] DELETE /identity/v3/projects/185aa1c5f816447e8d19cfb8c8ca9fbc => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:05.410220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.410941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.410941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.431949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:05.433668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:05.514225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 949/1889] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:05.518522 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.518834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.519116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.557257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:05.558826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:05.568021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 941/1890] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:05.576962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.587907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc67ba-3bb4-4f6a-9059-e52500ba2cad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.593919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.594783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.595631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.595631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.596480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:05.597491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.599472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.614597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 950/1891] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:05.619243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.628857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33798da5-99c6-478b-ab23-a7abb0a8d8ee None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.633846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.634514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.634827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.635101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.635812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.637757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.738160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 942/1892] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:05 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:05.742998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.755427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503f63-58d6-4ef8-af36-c1fbb5ad41af admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.819871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.828894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.828894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.828894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.828894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.828894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.919293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 951/1893] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:05 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:05.923612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.934232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537599-53fb-405f-bb53-b818709b4930 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.939180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.939849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.940133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.940410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.941135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.942762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.954658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 943/1894] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:05.958887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:05.970007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d221-4747-4442-bb19-2cdeb429b929 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:05.975129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:05.975805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:05.976089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:05.976368 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:05.977184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:05.978834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40ddaf-1dda-4e56-8988-257378087fba admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:05.992910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 952/1895] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:05.994068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.004858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef375d8-e5f5-42e4-aa8d-f0089e3b3284 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.010104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.011106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.011389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.011664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] PATH_INFO: `/v3/projects/3113492af0ac4af89975daec139b6347/users/631e55570ff447448f9090eec227701e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.028226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3113492af0ac4af89975daec139b6347, user_id=631e55570ff447448f9090eec227701e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.031072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.107593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 944/1896] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:05 2022] PUT /identity/v3/projects/3113492af0ac4af89975daec139b6347/users/631e55570ff447448f9090eec227701e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:06.118048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.118342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.118714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.146954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] MFA Rules not processed for user `631e55570ff447448f9090eec227701e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:06.148148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:06.152703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:06.217072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 953/1897] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:06.221474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.287143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cce8d-9b2a-4fd3-96b7-38a8c8aabf8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.292512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.293177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.293463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.293744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.294446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.296395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.393946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 945/1898] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:06 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:06.398032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.410120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975a88d-e3c4-4906-af6c-2c4a5862acad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.467367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.468006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.468233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.468510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.469225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.471251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.558932 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 954/1899] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:06 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:06.563967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.575065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafbae-d936-45bf-bd59-d7ac6ef0fa44 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.581520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.582209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.582495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.582811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.583545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.585462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.602605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 946/1900] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:06.605003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.622095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb5b6-06d3-44ef-a8f5-74ad10410749 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.628984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.629969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.630257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.630532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] PATH_INFO: `/v3/projects/e1c8144768f949f1a86f03895bc1bc55/users/9023f55b3be54dd9bdbf9c934fa8cf25/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.647666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c8144768f949f1a86f03895bc1bc55, user_id=9023f55b3be54dd9bdbf9c934fa8cf25, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.651296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.762212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 955/1901] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:06 2022] PUT /identity/v3/projects/e1c8144768f949f1a86f03895bc1bc55/users/9023f55b3be54dd9bdbf9c934fa8cf25/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:06.766585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.778913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ee9f-4810-46e7-b163-6c1374c7d9c8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.822371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:06.869379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.870674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.871074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.871599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.872425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.874265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db557c4-f331-4288-a653-d0afda31170d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.890612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 947/1902] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:06.894302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:06.928812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34be7e-ddbd-4296-8617-bdde00bd2440 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:06.941052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:06.941973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.944838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.945202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] PATH_INFO: `/v3/projects/e1c8144768f949f1a86f03895bc1bc55/users/9023f55b3be54dd9bdbf9c934fa8cf25/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:06.952923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c8144768f949f1a86f03895bc1bc55, user_id=9023f55b3be54dd9bdbf9c934fa8cf25, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:06.955619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:06.986627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 956/1903] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:06 2022] PUT /identity/v3/projects/e1c8144768f949f1a86f03895bc1bc55/users/9023f55b3be54dd9bdbf9c934fa8cf25/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:06.998315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86cef-7c4a-422b-9de0-4f10ecd626f5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:06.999423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86cef-7c4a-422b-9de0-4f10ecd626f5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:06.999423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86cef-7c4a-422b-9de0-4f10ecd626f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.053523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5ad86cef-7c4a-422b-9de0-4f10ecd626f5 None None] MFA Rules not processed for user `9023f55b3be54dd9bdbf9c934fa8cf25`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:07.055799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5ad86cef-7c4a-422b-9de0-4f10ecd626f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.129233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:07.131213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.180299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 948/1904] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:07.182509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f8a9-4e7f-4d53-93e7-2f6f5028225a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:07.183167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f8a9-4e7f-4d53-93e7-2f6f5028225a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:07.183448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f8a9-4e7f-4d53-93e7-2f6f5028225a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.184781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 949/1905] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:47:07.192032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:07.192395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:07.192676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.208848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa63634-e225-4c21-a0cc-eea2b152ebc8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:07.213011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:07.213667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:07.214282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:07.214411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.216264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.217084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:07.218258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.228108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:07.285357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:07.287744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:07.298590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 957/1906] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:07.299563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:07.299677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 950/1907] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:07.301084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.313606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e039609-2954-4643-8db7-968f12c5b3dd service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:07.321009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:07.321009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:07.321009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:07.321009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.328282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:07.330365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:07.342086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:07.344131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:07.353777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e5d6a-4f3a-47cc-a0a3-47dc6a7499d8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:07.356372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 958/1908] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:07.359972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:07.360714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:07.361001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:07.361274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:07.372595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:07.374264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:07.386908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 951/1909] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:08.540883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:08.550511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebddde-5544-4671-a29b-fc95b61ad795 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:08.555299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9023f55b3be54dd9bdbf9c934fa8cf25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1c8144768f949f1a86f03895bc1bc55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:08.556054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:08.556560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:08.556999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:08.558001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:08.558317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:08.559959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:08.588929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 959/1910] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:08.593772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:08.609699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263bc9e-5299-4610-90f9-f6544eff4107 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:08.617260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9023f55b3be54dd9bdbf9c934fa8cf25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1c8144768f949f1a86f03895bc1bc55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:08.618318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:08.618793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:08.619339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:08.620426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:08.622612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:08.726685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 952/1911] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:08 2022] POST /identity/v3/projects => generated 354 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:08.730701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:08.750409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb556ff-3e9c-4d76-bde0-e179b84a4c7e tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:08.867438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9023f55b3be54dd9bdbf9c934fa8cf25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1c8144768f949f1a86f03895bc1bc55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:08.868882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:08.869243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:08.869643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:08.870609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:08.870994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:08.875818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:08.888242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 960/1912] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:09.152987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:09.171087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fcfb3-4b97-4c46-ba69-7d5c37e44471 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:09.178227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9023f55b3be54dd9bdbf9c934fa8cf25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1c8144768f949f1a86f03895bc1bc55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:09.181214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:09.181214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:09.181214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] PATH_INFO: `/v3/projects/a2f96401a1024a48819f4f8d8e2e0a39` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:09.182863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a2f96401a1024a48819f4f8d8e2e0a39)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:09.185787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:09.194192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:09.387117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f96401a1024a48819f4f8d8e2e0a39', 'request_id': 'req-677ece22-ae7a-475c-8f4f-b9164e31afd3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:09.468146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f96401a1024a48819f4f8d8e2e0a39', 'request_id': 'req-677ece22-ae7a-475c-8f4f-b9164e31afd3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:09.516122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f96401a1024a48819f4f8d8e2e0a39', 'request_id': 'req-677ece22-ae7a-475c-8f4f-b9164e31afd3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:09.549463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] The token cache is being invalidate because project a2f96401a1024a48819f4f8d8e2e0a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:09.551579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:09.557859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-677ece22-ae7a-475c-8f4f-b9164e31afd3 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:09.559836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 953/1913] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:09 2022] DELETE /identity/v3/projects/a2f96401a1024a48819f4f8d8e2e0a39 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:10.341576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:10.344204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:10.457100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddc4c-30b5-400c-9dc5-25a34e945849 tempest-VolumeQuotaClassesTest-143433836 tempest-VolumeQuotaClassesTest-143433836-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:10.507662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:10.604939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:10.605947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:10.606233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:10.606509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] PATH_INFO: `/v3/users/631e55570ff447448f9090eec227701e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:10.608134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631e55570ff447448f9090eec227701e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:10.610125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:10.866394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631e55570ff447448f9090eec227701e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:10.873641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631e55570ff447448f9090eec227701e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:10.874754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631e55570ff447448f9090eec227701e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:10.875027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631e55570ff447448f9090eec227701e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:10.997350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631e55570ff447448f9090eec227701e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.025108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 961/1914] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:10 2022] DELETE /identity/v3/users/631e55570ff447448f9090eec227701e => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:11.030400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:11.030400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:11.030400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:11.050965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:11.052395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:11.130080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 954/1915] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:11.144443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:11.144443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:11.222516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927a06-4c31-4eb8-a8a9-e295ce548a06 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:11.226241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:11.227280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:11.227568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:11.227843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:11.236651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:11.238062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:11.254942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 962/1916] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:11.673738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:11.675186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:11.692364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8a07-6893-4997-a5b7-d4513ab6ef33 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:11.697665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:11.698576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:11.698862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:11.699190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] PATH_INFO: `/v3/projects/3113492af0ac4af89975daec139b6347` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:11.700446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3113492af0ac4af89975daec139b6347)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:11.702452 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:11.830088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3113492af0ac4af89975daec139b6347', 'request_id': 'req-50d8ff63-47b2-49c0-8b55-e90976dc832f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.902017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3113492af0ac4af89975daec139b6347', 'request_id': 'req-50d8ff63-47b2-49c0-8b55-e90976dc832f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.909071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3113492af0ac4af89975daec139b6347', 'request_id': 'req-50d8ff63-47b2-49c0-8b55-e90976dc832f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.922835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] The token cache is being invalidate because project 3113492af0ac4af89975daec139b6347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:11.923199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.924162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:11.926171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 955/1917] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:11 2022] DELETE /identity/v3/projects/3113492af0ac4af89975daec139b6347 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:11.940264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:11.943870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c33050-0373-4298-95a4-49befdcc934c service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:12.001551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:12.018906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:12.019811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:12.020105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:12.020629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] PATH_INFO: `/v3/users/9023f55b3be54dd9bdbf9c934fa8cf25` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:12.021817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9023f55b3be54dd9bdbf9c934fa8cf25)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:12.023813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:12.226173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9023f55b3be54dd9bdbf9c934fa8cf25'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:12.234255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9023f55b3be54dd9bdbf9c934fa8cf25'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:12.235007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9023f55b3be54dd9bdbf9c934fa8cf25'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:12.235631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9023f55b3be54dd9bdbf9c934fa8cf25'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:12.324026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8434de3a-5231-4d34-86bd-4005edb34a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9023f55b3be54dd9bdbf9c934fa8cf25'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:12.355033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 963/1918] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:11 2022] DELETE /identity/v3/users/9023f55b3be54dd9bdbf9c934fa8cf25 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:12.760252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:12.768133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:12.775891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ff63-47b2-49c0-8b55-e90976dc832f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:12.791173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:12.851437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:12.852530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:12.853502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:12.854107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] PATH_INFO: `/v3/projects/e1c8144768f949f1a86f03895bc1bc55` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:12.855498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c8144768f949f1a86f03895bc1bc55)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:12.857560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.042173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c8144768f949f1a86f03895bc1bc55', 'request_id': 'req-dd543ddd-9ca8-439d-93c6-7ddc6925e454'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:13.146659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c8144768f949f1a86f03895bc1bc55', 'request_id': 'req-dd543ddd-9ca8-439d-93c6-7ddc6925e454'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:13.155261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c8144768f949f1a86f03895bc1bc55', 'request_id': 'req-dd543ddd-9ca8-439d-93c6-7ddc6925e454'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:13.172152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] The token cache is being invalidate because project e1c8144768f949f1a86f03895bc1bc55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:13.172451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:13.173139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-dd543ddd-9ca8-439d-93c6-7ddc6925e454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:13.178097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 956/1919] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:12 2022] DELETE /identity/v3/projects/e1c8144768f949f1a86f03895bc1bc55 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:13.179765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.180204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.180452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.202021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:13.203896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:13.319412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 964/1920] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:13.336730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.336730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.336730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.399199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:13.401526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:13.414907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 957/1921] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:13.419623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.434716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8754d-f44f-4b50-a8a3-bca526cfcc67 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.458458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.459161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.459468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.459775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.460511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:13.460818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.463008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.492870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 965/1922] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:13.497104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.511628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea6a5d-c1a8-4263-b3a1-9f354c4a7278 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.517757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.518922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.519445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.519963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.521030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.523566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.640788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 958/1923] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:13 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:13.645105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.657253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12429a-e69c-4349-86d4-b68a1630ac4f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.710186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:13.720818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.721770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.722067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.722343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.723110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.725036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.828147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 966/1924] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:13 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:13.832185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.844375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89159d-9578-40ca-8da8-f328a59b7646 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.851616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.852409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.852409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.852605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.853592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.855404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.867258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 959/1925] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:13.871883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.881758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a9cdc-480e-4b53-82d2-7544fdeea25e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.887895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.888087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.888367 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.888643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.889383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.891011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9128f-454c-46c5-be98-849db71dd83d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:13.901862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 967/1926] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:13.906101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:13.916670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c3aa-678c-475e-a1b6-9bbae8378abf admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:13.922961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:13.922961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:13.922961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:13.923224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] PATH_INFO: `/v3/projects/b1694069a8304c8d80e9ca8721d423f7/users/fa5566a5bd164e589ff72ec5e5f388a4/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:13.939045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1694069a8304c8d80e9ca8721d423f7, user_id=fa5566a5bd164e589ff72ec5e5f388a4, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:13.941945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.028506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 960/1927] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:13 2022] PUT /identity/v3/projects/b1694069a8304c8d80e9ca8721d423f7/users/fa5566a5bd164e589ff72ec5e5f388a4/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:14.049546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.049546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.049546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.092072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] MFA Rules not processed for user `fa5566a5bd164e589ff72ec5e5f388a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:14.093446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:14.184613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 968/1928] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:14.189251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.242007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:14.258913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ccd0-98ab-468b-8540-0cbabad042b7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.263864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.264602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.264922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.265238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.265983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.267991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.390364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 961/1929] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:14 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:14.391676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.405362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9be05-d541-4d35-98c6-dbe1c968429c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.487278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.488038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.488493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.488615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.489286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.495603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.497172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.500310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:14.545129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67f8e-e355-4dc4-964e-1ad1430cb3c0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.555339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.556584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.557250 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.557655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] PATH_INFO: `/v3/users/80f3aba426f44deca103331aa7f37019` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.560224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f3aba426f44deca103331aa7f37019)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.562385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.584447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 969/1930] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:14 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:14.589523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.612135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1095ee-160e-4c71-b6e9-b5bc3c2120c6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.620062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.621492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.621875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.622483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.623483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.625625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.644152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 970/1931] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:14.655097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.685225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4dcf8-0c85-4d9a-a051-d5ab78b521a0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.697862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.698819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.699078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.699362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] PATH_INFO: `/v3/projects/694b09bb2da54fe29eb9498105f6839e/users/1bb84aa5a2b1475baf9107f2ea909078/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.726014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694b09bb2da54fe29eb9498105f6839e, user_id=1bb84aa5a2b1475baf9107f2ea909078, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.732486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.843272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 971/1932] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:14 2022] PUT /identity/v3/projects/694b09bb2da54fe29eb9498105f6839e/users/1bb84aa5a2b1475baf9107f2ea909078/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:14.844927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.856949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f3aba426f44deca103331aa7f37019'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:14.881377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f3aba426f44deca103331aa7f37019'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:14.886724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:14.951866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c1861-6434-4c18-858a-5d297913fda6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.957511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.958244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:14.958524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:14.958841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:14.959569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:14.961854 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:14.974731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 972/1933] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:14.977236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:14.991298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e76f6-3c2b-4c37-b11e-b864c43536c8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:14.997984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:14.998883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.000976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.001303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] PATH_INFO: `/v3/projects/694b09bb2da54fe29eb9498105f6839e/users/1bb84aa5a2b1475baf9107f2ea909078/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.003610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694b09bb2da54fe29eb9498105f6839e, user_id=1bb84aa5a2b1475baf9107f2ea909078, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:15.006746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4dde-e521-4636-86ba-7db6b8c95499 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:15.018828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 973/1934] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:14 2022] PUT /identity/v3/projects/694b09bb2da54fe29eb9498105f6839e/users/1bb84aa5a2b1475baf9107f2ea909078/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:15.031199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.031484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.031798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.056795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f3aba426f44deca103331aa7f37019'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:15.057588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f3aba426f44deca103331aa7f37019'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:15.059350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ed4d4121-e459-43e9-9985-f5f67b0e9c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f3aba426f44deca103331aa7f37019'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:15.060605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] MFA Rules not processed for user `1bb84aa5a2b1475baf9107f2ea909078`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:15.061853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.079983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 962/1935] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:14 2022] DELETE /identity/v3/users/80f3aba426f44deca103331aa7f37019 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:15.084871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.085172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.085451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.122728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:15.124630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.194850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 974/1936] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:15.223294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:15.226355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.246379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 963/1937] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:15.257423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:15.259273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.298797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706117d-eabd-4574-a6c2-68339a83abb2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:15.353548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:15.363587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:15.364248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.364532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.364877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.367343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.376481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5071f7-68ad-4270-89d0-ef79d3e61eb4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:15.382075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:15.383154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.383644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.385171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.396647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:15.398243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8d4b-5517-46ce-8c1a-c2227cebbb84 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:15.410351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 964/1938] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:15.457937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:15.457937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1be9-73d5-4c32-90d9-cb874c70d11c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:15.467518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 975/1939] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:15.741946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.742236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.742512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.753439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 965/1940] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:47:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:47:15.762438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.762761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.763043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.805755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:15.808674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.912443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 976/1941] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:15.936098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:15.937873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:15.965193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adfb84-23cc-41db-a81f-fd1d02583945 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:15.971222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:15.972198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:15.972906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:15.972906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:15.986663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:15.988257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:16.011048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 966/1942] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:16.023814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:16.323782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:16.326921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:16.361254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed099a39-d541-4114-9d2a-58afe7902836 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:16.372232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:16.382237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:16.383685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:16.397283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] PATH_INFO: `/v3/projects/dc0e0891981748c3b34c68c65a3bc781` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:16.397283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0e0891981748c3b34c68c65a3bc781)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:16.397573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:16.590888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0e0891981748c3b34c68c65a3bc781', 'request_id': 'req-e5faf03f-7111-4773-884b-cd6ae36429c5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:16.735867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0e0891981748c3b34c68c65a3bc781', 'request_id': 'req-e5faf03f-7111-4773-884b-cd6ae36429c5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:16.743037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0e0891981748c3b34c68c65a3bc781', 'request_id': 'req-e5faf03f-7111-4773-884b-cd6ae36429c5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:16.768926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] The token cache is being invalidate because project dc0e0891981748c3b34c68c65a3bc781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:16.769232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:16.769919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:16.771572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 977/1943] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:16 2022] DELETE /identity/v3/projects/dc0e0891981748c3b34c68c65a3bc781 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:16.776937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:16.794587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51db356-b931-4e71-9a96-5849b34c70c0 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:16.888635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:16.889524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:16.889810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:16.890083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] PATH_INFO: `/v3/users/a8c639539c764964ac0d718361c299e5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:16.891334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c639539c764964ac0d718361c299e5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:16.894974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:17.167392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c639539c764964ac0d718361c299e5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:17.175576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c639539c764964ac0d718361c299e5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:17.220769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c639539c764964ac0d718361c299e5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:17.221679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c639539c764964ac0d718361c299e5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:17.222301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c639539c764964ac0d718361c299e5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:17.241718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 967/1944] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:16 2022] DELETE /identity/v3/users/a8c639539c764964ac0d718361c299e5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:17.766256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:17.769052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:17.813715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faf03f-7111-4773-884b-cd6ae36429c5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:17.826222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:17.832443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:17.834394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:17.962808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:17.963914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:17.964199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:17.964473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] PATH_INFO: `/v3/projects/b1994f9771eb4baab928b816c989f496` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:17.966106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:17.966785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1994f9771eb4baab928b816c989f496)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:17.968982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:17.981755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9f2a-cc3d-490a-87a3-157d0ce89db6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:17.989158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:17.990080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:17.998842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:18.006973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] PATH_INFO: `/v3/users/cc603224add247778c396d8ab4a8dcb1` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:18.006973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc603224add247778c396d8ab4a8dcb1)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:18.006973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:18.145896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1994f9771eb4baab928b816c989f496', 'request_id': 'req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.234352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1994f9771eb4baab928b816c989f496', 'request_id': 'req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.244702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1994f9771eb4baab928b816c989f496', 'request_id': 'req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.248936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc603224add247778c396d8ab4a8dcb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.256227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc603224add247778c396d8ab4a8dcb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.256568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] The token cache is being invalidate because project b1994f9771eb4baab928b816c989f496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:18.256865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.257525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.260078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 978/1945] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:17 2022] DELETE /identity/v3/projects/b1994f9771eb4baab928b816c989f496 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:18.263788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:18.266039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:18.329732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc603224add247778c396d8ab4a8dcb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.330755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc603224add247778c396d8ab4a8dcb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.331967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc603224add247778c396d8ab4a8dcb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.365871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00641bb2-5aaa-4300-8a58-b8d59ecf8e76 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:18.375458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 968/1946] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:17 2022] DELETE /identity/v3/users/cc603224add247778c396d8ab4a8dcb1 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:18.378332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:18.378332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:18.378332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:18.378332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] PATH_INFO: `/v3/users/28981d5e278e43388db1ef9c250802bd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:18.379424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28981d5e278e43388db1ef9c250802bd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:18.382024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:18.628642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28981d5e278e43388db1ef9c250802bd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.637361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28981d5e278e43388db1ef9c250802bd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.638319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28981d5e278e43388db1ef9c250802bd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.638944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28981d5e278e43388db1ef9c250802bd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.753312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28981d5e278e43388db1ef9c250802bd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:18.760470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:18.762190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:18.782623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 979/1947] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:18 2022] DELETE /identity/v3/users/28981d5e278e43388db1ef9c250802bd => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:18.866322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778edf-cca4-43b5-84e6-cb2104985370 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:18.935071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:18.936150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:18.936511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:18.936863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] PATH_INFO: `/v3/projects/d11a34d9d6824eca941763667839c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:18.938276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11a34d9d6824eca941763667839c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:18.940416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.088783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11a34d9d6824eca941763667839c78e', 'request_id': 'req-686a7df3-42e7-4b73-b8ab-6dd3897e5387'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.169494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11a34d9d6824eca941763667839c78e', 'request_id': 'req-686a7df3-42e7-4b73-b8ab-6dd3897e5387'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.176394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11a34d9d6824eca941763667839c78e', 'request_id': 'req-686a7df3-42e7-4b73-b8ab-6dd3897e5387'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.190263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] The token cache is being invalidate because project d11a34d9d6824eca941763667839c78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:19.191036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.191954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.198822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:19.202292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 969/1948] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:18 2022] DELETE /identity/v3/projects/d11a34d9d6824eca941763667839c78e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:19.202515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:19.217326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:19.219511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:19.228339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c57b5-df44-4f89-b4cd-ffb47cfc9ac8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:19.316793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7df3-42e7-4b73-b8ab-6dd3897e5387 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:19.326518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:19.327372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:19.327854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.328297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.328466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.328572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.330065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] PATH_INFO: `/v3/users/069843fb01d2404c931f062d6b2ec2b0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.330573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] PATH_INFO: `/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.332404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] RBAC: Authorizing `identity:delete_project(project_id=45eb6ef2612c4adf9edc10c07ff5545a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:19.333593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069843fb01d2404c931f062d6b2ec2b0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:19.335487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.339529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.524124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069843fb01d2404c931f062d6b2ec2b0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.543237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069843fb01d2404c931f062d6b2ec2b0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.544223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069843fb01d2404c931f062d6b2ec2b0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.545031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069843fb01d2404c931f062d6b2ec2b0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.554631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45eb6ef2612c4adf9edc10c07ff5545a', 'request_id': 'req-44e312e8-226a-4302-8572-b4cce235477c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.599119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069843fb01d2404c931f062d6b2ec2b0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.621590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 980/1949] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:19 2022] DELETE /identity/v3/users/069843fb01d2404c931f062d6b2ec2b0 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:19.622584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:19.627753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:19.660990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45eb6ef2612c4adf9edc10c07ff5545a', 'request_id': 'req-44e312e8-226a-4302-8572-b4cce235477c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.669613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45eb6ef2612c4adf9edc10c07ff5545a', 'request_id': 'req-44e312e8-226a-4302-8572-b4cce235477c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.684575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] The token cache is being invalidate because project 45eb6ef2612c4adf9edc10c07ff5545a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:19.685227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.686074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-44e312e8-226a-4302-8572-b4cce235477c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.697025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 970/1950] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:19 2022] DELETE /identity/v3/projects/45eb6ef2612c4adf9edc10c07ff5545a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:19.699139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c4b-68c9-433c-95f1-0ffe564f2691 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.699517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c4b-68c9-433c-95f1-0ffe564f2691 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.699902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c4b-68c9-433c-95f1-0ffe564f2691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.707640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7f021-9d9f-4bdd-952d-5726ffd43b81 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:19.715337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:19.716631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.717666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.718124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] PATH_INFO: `/v3/users/0643be0e26fb41b7adc76c759cea339d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.719580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0643be0e26fb41b7adc76c759cea339d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:19.730533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-eb474c4b-68c9-433c-95f1-0ffe564f2691 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:19.731614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.732127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-eb474c4b-68c9-433c-95f1-0ffe564f2691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:19.744638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 971/1951] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:19.753264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.754045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.755017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.799775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:19.801762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:19.818181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 972/1952] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:19.824174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:19.848338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a72647-46e8-4cf0-ae39-7f13495a5378 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:19.858085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:19.860113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.860495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.861010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.862677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:19.870674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:19.875672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:19.883058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.905385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 973/1953] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:47:19.906400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:19.920388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92e2-654c-4ae3-a11b-df728e2d36c9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:19.926508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:19.927859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:19.934600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:19.934600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:19.934821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0643be0e26fb41b7adc76c759cea339d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.934915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:19.940436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:19.943143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0643be0e26fb41b7adc76c759cea339d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.944567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0643be0e26fb41b7adc76c759cea339d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:19.945291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0643be0e26fb41b7adc76c759cea339d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.057101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0643be0e26fb41b7adc76c759cea339d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.058442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:20.067602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 974/1954] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:19 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:20.068702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.070990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.099436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 981/1955] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:19 2022] DELETE /identity/v3/users/0643be0e26fb41b7adc76c759cea339d => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:20.105683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.111564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.192949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5497a-e04a-42c4-ad64-f2fe7cdf008c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.236026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07859-54e5-4764-9f73-2ea00cdea067 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.242110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.242817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.243098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.246875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.246875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.246875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09aa91-0c99-486b-ad1d-d99f5c98ae80 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.266536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.268188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.268554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.268888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] PATH_INFO: `/v3/projects/3c88d44c963e4ad4b4915bdcea315848` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.270194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c88d44c963e4ad4b4915bdcea315848)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.272114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.333329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 982/1956] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:20.333329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.333329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.333329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.350307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:20.351549 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.359791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 983/1957] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:20.367374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.370111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5f005-1796-41bb-95a7-7080ef1c6e4b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.375131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.375801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.376087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.376364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.377112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.378743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.389930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 984/1958] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:20.390948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.392941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.437904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c88d44c963e4ad4b4915bdcea315848', 'request_id': 'req-159018af-3072-4bbe-a882-f49677b3a248'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.482406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a28-9a1c-4142-96d8-95e470ddc89e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.512401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c88d44c963e4ad4b4915bdcea315848', 'request_id': 'req-159018af-3072-4bbe-a882-f49677b3a248'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.527483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c88d44c963e4ad4b4915bdcea315848', 'request_id': 'req-159018af-3072-4bbe-a882-f49677b3a248'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.544123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] The token cache is being invalidate because project 3c88d44c963e4ad4b4915bdcea315848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:20.544123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.544767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:20.547516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 975/1959] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:20 2022] DELETE /identity/v3/projects/3c88d44c963e4ad4b4915bdcea315848 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:20.548612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.551674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.565083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.565752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.566037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.566322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.572857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.633079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159018af-3072-4bbe-a882-f49677b3a248 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.638058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.638754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.639080 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.639397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.640132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.641801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcdcb3-6c2a-4628-9f9b-e723346d4e63 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.643312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.644764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.659108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 976/1960] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:20.659108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28728-70f4-4751-b22c-98916af7134a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.659359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28728-70f4-4751-b22c-98916af7134a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.659430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28728-70f4-4751-b22c-98916af7134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.662717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 985/1961] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:20.663467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.678077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62da954-819e-477e-8149-b355b5a2c53e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.684331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.685224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.685505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.685835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] PATH_INFO: `/v3/projects/87eb037c16464a9887edc080790f29eb/users/7c88f8bcf566470db866540c2bbb12ce/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.694318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e5e28728-70f4-4751-b22c-98916af7134a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:20.695781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e5e28728-70f4-4751-b22c-98916af7134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.704453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 977/1962] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:20.711167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87eb037c16464a9887edc080790f29eb, user_id=7c88f8bcf566470db866540c2bbb12ce, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.714601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e78cb1-76fc-4883-ad3e-fed74c73caa1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.721028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.721973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.722316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.742227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:20.743660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.752781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 978/1963] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:20.756579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.765720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043de6a-7ea9-4967-ba9d-ab6314b8a20d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.770301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.772739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.772739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.772739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.772739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:20.772739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.774935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.790167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 979/1964] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:47:20.791876 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:20.799619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 986/1965] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:20 2022] PUT /identity/v3/projects/87eb037c16464a9887edc080790f29eb/users/7c88f8bcf566470db866540c2bbb12ce/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:20.804243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5007f-0ab6-4c9e-9873-691821368d15 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:20.811542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.812013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.812309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.842048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] MFA Rules not processed for user `7c88f8bcf566470db866540c2bbb12ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:20.843767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:20.880471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:20.881156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:20.881473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:20.881771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:20.882531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:20.885485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:20.984802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 987/1966] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:20.997481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 980/1967] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:20 2022] POST /identity/v3/projects => generated 361 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:21.002608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.002747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.004801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:21.020019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eefee7-2ddb-4be3-a59f-9be744793a7e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.068130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:21.098962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.099987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.100366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.100722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.101489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.103843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:21.121502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546804d-9a31-4a16-a2ec-d6f3c81d16b5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.126418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.127559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.127559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.127789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.176778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 988/1968] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:21 2022] POST /identity/v3/users => generated 281 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:21.181052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.192278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060626c-d3b1-46de-b61a-be6fc43c23ee admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.199285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.200226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.200653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.201197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.202097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.203922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:21.206490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.207884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:21.219893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 989/1969] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:21.220182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 981/1970] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:21.220993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.235139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdfc93-1611-47a6-8020-23f9d3725c16 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.241086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.241975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.242184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.242477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.243798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.248670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19d282-f4c0-4342-9dbd-6264347b051c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:21.263066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 990/1971] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:21.264257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.277260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3b00a-28d2-4d70-9cc2-5f6fdf2764f5 service glance] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.282761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.284131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.284592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.295756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] PATH_INFO: `/v3/projects/35be8d699d584ab2a6d3e2d8cd91067c/users/054317f4ebc54adbb9306b95f6c33dd2/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.323074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=35be8d699d584ab2a6d3e2d8cd91067c, user_id=054317f4ebc54adbb9306b95f6c33dd2, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.326398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:21.332570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:21.440308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 982/1972] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:21 2022] PUT /identity/v3/projects/35be8d699d584ab2a6d3e2d8cd91067c/users/054317f4ebc54adbb9306b95f6c33dd2/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:21.451880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.452171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.452443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.488212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] MFA Rules not processed for user `054317f4ebc54adbb9306b95f6c33dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:21.489400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:21.575294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 991/1973] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:21.756791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:21.756791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:21.907791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60473-5f7d-4fd1-8702-2c7348ad7f3c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:21.933798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:21.935073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:21.935254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:21.935531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] PATH_INFO: `/v3/projects/6047a462ca234458a329166d92855012` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:21.937377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6047a462ca234458a329166d92855012)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:21.938964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:22.061149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6047a462ca234458a329166d92855012', 'request_id': 'req-b8004aca-d4ed-4dac-af3b-baf0f1081961'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.147629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6047a462ca234458a329166d92855012', 'request_id': 'req-b8004aca-d4ed-4dac-af3b-baf0f1081961'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.152296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:22.164722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6047a462ca234458a329166d92855012', 'request_id': 'req-b8004aca-d4ed-4dac-af3b-baf0f1081961'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.211132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] The token cache is being invalidate because project 6047a462ca234458a329166d92855012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:22.212117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.213386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.216455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 983/1974] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:21 2022] DELETE /identity/v3/projects/6047a462ca234458a329166d92855012 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:22.220599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:22.234482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0f69-6c3c-4579-949c-169f1aa28990 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:22.308812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:22.310707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:22.328062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:22.329623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:22.330156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:22.330974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] PATH_INFO: `/v3/users/710915bd7bac4d76adad8575e240fc0a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:22.333143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=710915bd7bac4d76adad8575e240fc0a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:22.335515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:22.389111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8004aca-d4ed-4dac-af3b-baf0f1081961 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:22.392809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:22.393449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:22.393726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:22.393994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:22.395855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:22.455463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:22.457719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:22.468431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 984/1975] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:22.469391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:22.470730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:22.482378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b548-ceb2-40e9-a1a9-de041f549c72 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:22.485698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:22.486355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:22.486661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:22.486935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:22.488795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:22.533363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710915bd7bac4d76adad8575e240fc0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.541392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710915bd7bac4d76adad8575e240fc0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.542459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710915bd7bac4d76adad8575e240fc0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.543327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710915bd7bac4d76adad8575e240fc0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.564383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:22.565862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:22.577944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 985/1976] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:22.578861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:22.580325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:22.596444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6379-13be-452f-8e7a-940d28909f68 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:22.601423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:22.602347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:22.602697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:22.602983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] PATH_INFO: `/v3/users/054317f4ebc54adbb9306b95f6c33dd2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:22.604517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054317f4ebc54adbb9306b95f6c33dd2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:22.606518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:22.620748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:22.648745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f0e081-7c7a-4cef-b182-689b1bc603cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710915bd7bac4d76adad8575e240fc0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.679881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 992/1977] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:22 2022] DELETE /identity/v3/users/710915bd7bac4d76adad8575e240fc0a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:22.828994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054317f4ebc54adbb9306b95f6c33dd2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.837722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054317f4ebc54adbb9306b95f6c33dd2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.942221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054317f4ebc54adbb9306b95f6c33dd2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.944696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054317f4ebc54adbb9306b95f6c33dd2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.945550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054317f4ebc54adbb9306b95f6c33dd2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:22.975791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 986/1978] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:22 2022] DELETE /identity/v3/users/054317f4ebc54adbb9306b95f6c33dd2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:22.980416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:22.980719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:22.981005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.001277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:23.002695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.090074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 993/1979] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:23.100454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.101923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.132898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.134935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.166935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677858d-0533-436b-b680-a2721ab89f77 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.172378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.176538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.176538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.176538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] PATH_INFO: `/v3/projects/24606728b9b24894abb5e7652e3054a3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.176538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24606728b9b24894abb5e7652e3054a3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.178906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:23.219823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad713ae-ad24-4e39-acc8-8845cde917b9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.234202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.234926 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.235221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.235503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.249751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.253145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaed540-72b3-4216-a39d-6e6c49c2a0ba service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:23.294347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24606728b9b24894abb5e7652e3054a3', 'request_id': 'req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:23.303072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 987/1980] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:23.407443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24606728b9b24894abb5e7652e3054a3', 'request_id': 'req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:23.410634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:23.415710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24606728b9b24894abb5e7652e3054a3', 'request_id': 'req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:23.452161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] The token cache is being invalidate because project 24606728b9b24894abb5e7652e3054a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:23.452694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:23.453635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7b0fafbc-dbb9-4f17-8fe3-01edb8bcddd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:23.456298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 994/1981] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:23 2022] DELETE /identity/v3/projects/24606728b9b24894abb5e7652e3054a3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:23.463071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.464713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.467073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.510397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:23.512008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.589768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 988/1982] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:23.598048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ef811-9c03-4603-bebb-436639f254be None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.598333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ef811-9c03-4603-bebb-436639f254be None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.598859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ef811-9c03-4603-bebb-436639f254be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.625042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-477ef811-9c03-4603-bebb-436639f254be None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:23.626506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-477ef811-9c03-4603-bebb-436639f254be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.638452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 995/1983] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:23.642940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.654483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac8ed8-941c-4477-969b-e24c0dd479fb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.660988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.661762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.662111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.662451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.663900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:23.665063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.667975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:23.682712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 989/1984] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:23.687822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ef811-9c03-4603-bebb-436639f254be None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.699562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ef811-9c03-4603-bebb-436639f254be None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.704568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.705250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.705558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.705837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.706568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.708503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:23.746484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:23.841146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 996/1985] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:23 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:23.846944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.847126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:23.848954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:23.861379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f554f36-b08b-4322-8455-d12517ef5a66 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.929388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5896b1f-2d9c-4efe-9659-073b44bf9188 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:23.932335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.933006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.933288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.933571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.934312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.936412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:23.936623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:23.937481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:23.937794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:23.938075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] PATH_INFO: `/v3/projects/35be8d699d584ab2a6d3e2d8cd91067c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:23.939302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35be8d699d584ab2a6d3e2d8cd91067c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:23.941198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.016271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 990/1986] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:23 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:24.020128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.030081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef530a-a217-45a6-b9f0-281160503d15 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.034962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.035780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.037192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.037473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.043435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.043435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.051032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 991/1987] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:24.055067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.056537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35be8d699d584ab2a6d3e2d8cd91067c', 'request_id': 'req-1e1ae640-fd5d-419a-b92f-033c248bd879'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:24.065478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643a706-cabd-43ef-ab0a-056d5786f1e1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.123329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.123914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.124196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.124511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.125328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.126993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.138379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 992/1988] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:24.142677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.153859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979962b3-7e9d-4ccf-8f4e-7b24712de661 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.157843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35be8d699d584ab2a6d3e2d8cd91067c', 'request_id': 'req-1e1ae640-fd5d-419a-b92f-033c248bd879'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:24.158813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.159711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.160003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.160283 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] PATH_INFO: `/v3/projects/a94e83d2571949c18031b0a2df3f5949/users/c8eb5ec4825741898e49baf5f8583fcf/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.165299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35be8d699d584ab2a6d3e2d8cd91067c', 'request_id': 'req-1e1ae640-fd5d-419a-b92f-033c248bd879'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:24.176852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] The token cache is being invalidate because project 35be8d699d584ab2a6d3e2d8cd91067c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:24.177463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:24.178009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e1ae640-fd5d-419a-b92f-033c248bd879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:24.179837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 997/1989] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:23 2022] DELETE /identity/v3/projects/35be8d699d584ab2a6d3e2d8cd91067c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:24.184257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94e83d2571949c18031b0a2df3f5949, user_id=c8eb5ec4825741898e49baf5f8583fcf, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.184908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147c37-be3e-44c7-baec-bbb7996a5deb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.185201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147c37-be3e-44c7-baec-bbb7996a5deb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.190384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147c37-be3e-44c7-baec-bbb7996a5deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.190502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11d273-56d7-4e74-b788-a95cd7cea960 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.204710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-84147c37-be3e-44c7-baec-bbb7996a5deb None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:24.205904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84147c37-be3e-44c7-baec-bbb7996a5deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:24.214862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 998/1990] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:24.226211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.226583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.226895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.248565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:24.253096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:24.262396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 999/1991] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:24.266876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.279136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71840431-c6db-4997-8d3e-07669587ec94 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.280277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 993/1992] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:24 2022] PUT /identity/v3/projects/a94e83d2571949c18031b0a2df3f5949/users/c8eb5ec4825741898e49baf5f8583fcf/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:24.292323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.292727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.292899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.318992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:24.323212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] MFA Rules not processed for user `c8eb5ec4825741898e49baf5f8583fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:24.324459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:24.356857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.357997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.357997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.358249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.359046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:24.359345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.361262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.378850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1000/1993] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:47:24.391809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.408044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fa7a8-5b5f-4eea-ac70-d30dfdc23a02 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.414065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.414995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.415881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.416219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.416748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.418756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.432285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 994/1994] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:24.437541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.449178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:24.460499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618edf29-b656-4bb6-ae21-edf2ec916c98 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.467682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.468520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.468809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.469108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.469815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.478348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.551901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1001/1995] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:24.560182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.572529 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 995/1996] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:24.576666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.592806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:24.640825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163cc8e-9e39-4190-8824-94014f5752c9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.646973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.647195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.647481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.647759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.648456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.651112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.652814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e89885-f43c-49b9-a26c-07e4290e916d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.659358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.660605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.661548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.662061 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.664634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.667244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.730667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1002/1997] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:24.735290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.745494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41b18a-e1ca-4eb6-bcca-b655e9dc768d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.756697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.756838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 996/1998] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:24.756933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.756933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.756933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.756933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.756933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.761139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.770336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1003/1999] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:24.771755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aeb64-7eb2-49e3-98cb-c355c3ddddc1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.775409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.777738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.778608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.779106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.779302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.780013 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.781615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.786513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75924c5-cc36-4e2e-813b-ba0842d13816 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.792755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.793426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.797189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.797313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 997/2000] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:24.797418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.797418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.797418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.798088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.809231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cc269-fc25-4837-ab3f-7b35ba2e9cab admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.811100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1004/2001] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:24.814276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.815133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.815287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.815577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.816415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] PATH_INFO: `/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2/users/dc40853b79424df59dc99a7e1fcca257/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.825369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e1fe-3b9b-4527-98a2-4c30bbfb0174 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:24.830191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:24.831136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.831415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.831689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] PATH_INFO: `/v3/projects/b0f367d309d74e7eaaa23d732fc7c65a/users/0e3c4968462c4cfe9824e0947813f635/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.832961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7b0a8dff604290a73c6d562ca17ac2, user_id=dc40853b79424df59dc99a7e1fcca257, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.835712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b599f67-5649-4940-8a93-514e5ee9baf3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.847216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f367d309d74e7eaaa23d732fc7c65a, user_id=0e3c4968462c4cfe9824e0947813f635, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:24.850087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:24.958620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1005/2002] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:24 2022] PUT /identity/v3/projects/b0f367d309d74e7eaaa23d732fc7c65a/users/0e3c4968462c4cfe9824e0947813f635/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:24.959037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 998/2003] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:24 2022] PUT /identity/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2/users/dc40853b79424df59dc99a7e1fcca257/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:24.963581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:24.970384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:24.970719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:24.971070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:24.999153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] MFA Rules not processed for user `0e3c4968462c4cfe9824e0947813f635`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:25.001396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.030730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0ba98-7eef-47f4-b098-83d1111231be admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.075428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 999/2004] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:25.090670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:25.092186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.095735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:25.097668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.098339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.098682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.098968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.099700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.101299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.112167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1006/2005] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:25.116297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:25.126827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec92b3-8d4c-43e0-b6ce-10ff28d144ec admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.132868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.134432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.134770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.135046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] PATH_INFO: `/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2/users/dc40853b79424df59dc99a7e1fcca257/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.137207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7b0a8dff604290a73c6d562ca17ac2, user_id=dc40853b79424df59dc99a7e1fcca257, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.139912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cec34c-6a70-4565-9a68-549499d58b87 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.151285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1007/2006] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:25 2022] PUT /identity/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2/users/dc40853b79424df59dc99a7e1fcca257/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:25.158302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ae8da-7bb7-4a79-9e59-6d4a26d815f7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.161570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.161863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.162145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.194028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] MFA Rules not processed for user `dc40853b79424df59dc99a7e1fcca257`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:25.195314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.217235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.217826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.218153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.218432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.233308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:25.268758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1008/2007] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:25.280428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:25.281868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.283538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.285101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.295006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bcc8d-dc8d-4de8-98fd-1bc7bebdf854 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.295185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1000/2008] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:25.298882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.299712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.299995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.300273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.311275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.312739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7154d-bc30-46d7-878a-fe3148757f30 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.322971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1009/2009] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:25.351902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:25.391993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:25.424198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea651-9684-4897-b300-5ff18ec07af6 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.429043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.429958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.431267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.431267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] PATH_INFO: `/v3/users/c8eb5ec4825741898e49baf5f8583fcf` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.432321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8eb5ec4825741898e49baf5f8583fcf)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.434239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.617349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8eb5ec4825741898e49baf5f8583fcf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:25.624327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8eb5ec4825741898e49baf5f8583fcf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:25.661380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:25.695093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8eb5ec4825741898e49baf5f8583fcf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:25.695777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8eb5ec4825741898e49baf5f8583fcf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:25.697065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8eb5ec4825741898e49baf5f8583fcf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:25.716156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1001/2010] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:25 2022] DELETE /identity/v3/users/c8eb5ec4825741898e49baf5f8583fcf => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:25.720721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.721456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.721874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.741481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:25.742721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.816387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1010/2011] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:25.828233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:25.830278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:25.892634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5e159-920e-4688-8095-861e53a93814 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:25.896633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:25.897289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:25.897570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:25.897843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:25.908470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:25.909962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:25.922246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1002/2012] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:26.194859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:26.422179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:26.423866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:26.436537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7470c36-e718-46d4-90de-ef91fe8627f2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:26.441408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:26.442331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:26.442895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:26.443326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] PATH_INFO: `/v3/projects/a94e83d2571949c18031b0a2df3f5949` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:26.444846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94e83d2571949c18031b0a2df3f5949)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:26.447714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:26.577273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94e83d2571949c18031b0a2df3f5949', 'request_id': 'req-b423af2f-fecc-4301-b182-17d2847bae55'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:26.587007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:26.668995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94e83d2571949c18031b0a2df3f5949', 'request_id': 'req-b423af2f-fecc-4301-b182-17d2847bae55'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:26.676786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94e83d2571949c18031b0a2df3f5949', 'request_id': 'req-b423af2f-fecc-4301-b182-17d2847bae55'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:26.688142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] The token cache is being invalidate because project a94e83d2571949c18031b0a2df3f5949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:26.688435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:26.689097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:26.690946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1011/2013] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:26 2022] DELETE /identity/v3/projects/a94e83d2571949c18031b0a2df3f5949 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:26.706281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:26.717249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e819-8a8e-49bd-93d1-79917cdce682 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:26.752025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:26.779690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:26.784024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:26.784024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:26.784024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] PATH_INFO: `/v3/users/dc40853b79424df59dc99a7e1fcca257` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:26.784024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc40853b79424df59dc99a7e1fcca257)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:26.784730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:27.007606 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc40853b79424df59dc99a7e1fcca257'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.014930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc40853b79424df59dc99a7e1fcca257'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.077712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc40853b79424df59dc99a7e1fcca257'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.078652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc40853b79424df59dc99a7e1fcca257'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.079493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c1cbef44-4233-4e2e-ba1f-9aefc5fe0d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc40853b79424df59dc99a7e1fcca257'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.105216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1003/2014] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:26 2022] DELETE /identity/v3/users/dc40853b79424df59dc99a7e1fcca257 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:27.188106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:27.674596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:27.677559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:27.692614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423af2f-fecc-4301-b182-17d2847bae55 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:27.757162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:27.757993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:27.758285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:27.758614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] PATH_INFO: `/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:27.761653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7b0a8dff604290a73c6d562ca17ac2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:27.762642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:27.909347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7b0a8dff604290a73c6d562ca17ac2', 'request_id': 'req-4c8d6d98-a36b-472f-a3a4-48068989c399'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.974866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7b0a8dff604290a73c6d562ca17ac2', 'request_id': 'req-4c8d6d98-a36b-472f-a3a4-48068989c399'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.981607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7b0a8dff604290a73c6d562ca17ac2', 'request_id': 'req-4c8d6d98-a36b-472f-a3a4-48068989c399'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.993647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] The token cache is being invalidate because project 8d7b0a8dff604290a73c6d562ca17ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:27.994106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.994708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4c8d6d98-a36b-472f-a3a4-48068989c399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:27.999782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1012/2015] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:27 2022] DELETE /identity/v3/projects/8d7b0a8dff604290a73c6d562ca17ac2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:28.001503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.002006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.002094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.023071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:28.024368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:28.058226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:28.096847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1004/2016] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:28.105340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.108559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.108559 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.128078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:28.130139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:28.142935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1013/2017] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:28.144776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.156207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3467f-e83a-4b62-b7a8-daaf3f33f1d5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.163573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.165909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.165909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.165909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.165909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:28.166224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.167901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.185070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1005/2018] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:28.186886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.201147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585be8e-64f8-45ee-b81e-279ce6ae26e3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.207915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.208698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.209060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.209415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.210572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.213458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.231297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:28.315486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.319114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1014/2019] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:28 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:28.331504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50539b7-8c60-4db6-9040-93175f8492c0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.412871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.413553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.413851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.414172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.415360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.417341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.551839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1006/2020] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:28 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:28.556871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.570628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0489ab-74fd-4bf2-bcde-062fc73f883b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.575473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.576151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.576437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.576714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.577434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.579078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.593110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1015/2021] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:28.594332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.606311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967753c-eb8d-4cff-841b-338b6c5d59a1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.610237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.611039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.611395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.611744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.612484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.614096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8793e8c-921f-463a-83f9-644c0e212de7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.628689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1007/2022] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:28.630161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.640393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97397b9c-7c63-4cb0-a7d0-ea5cf1653f63 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.645460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.646351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.646665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.646944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] PATH_INFO: `/v3/projects/cc189f2f94f64882a68608d125f4d248/users/bb1ebfb16aa54fb5826ec6dacac9233c/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.662338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc189f2f94f64882a68608d125f4d248, user_id=bb1ebfb16aa54fb5826ec6dacac9233c, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.665564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.731479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1016/2023] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:28 2022] PUT /identity/v3/projects/cc189f2f94f64882a68608d125f4d248/users/bb1ebfb16aa54fb5826ec6dacac9233c/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:28.745057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.745266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.745548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.779299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] MFA Rules not processed for user `bb1ebfb16aa54fb5826ec6dacac9233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:28.780755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:28.857318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1008/2024] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:28.858596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.978821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:28.980356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:28.981077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dfd8-6105-4edd-9338-1514305baee2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:28.986986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:28.987764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:28.988395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:28.988776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:28.989582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:28.992423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:28.995990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fa910-2cce-430c-b649-0873de8ebee1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.002992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.004293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.004584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.004859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] PATH_INFO: `/v3/users/fa5566a5bd164e589ff72ec5e5f388a4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.007196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5566a5bd164e589ff72ec5e5f388a4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.012010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.093435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1017/2025] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:28 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:29.093435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.179343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bbabe-7bda-4e67-909a-80ee3d5d7677 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.184826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.185509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.185794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.186073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.187192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.195153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.209620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5566a5bd164e589ff72ec5e5f388a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:29.217272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5566a5bd164e589ff72ec5e5f388a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:29.260528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1018/2026] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:29 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:29.261383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.277775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660183a-fe91-4be3-a428-2996f6643473 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.282966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.283646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.283933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.284444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.285126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.286909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.289286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5566a5bd164e589ff72ec5e5f388a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:29.289988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5566a5bd164e589ff72ec5e5f388a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:29.290725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ac366358-a5bc-45b6-885b-92c49eecebc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5566a5bd164e589ff72ec5e5f388a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:29.311269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1009/2027] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:28 2022] DELETE /identity/v3/users/fa5566a5bd164e589ff72ec5e5f388a4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:29.314586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1019/2028] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:29.315841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.316190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.316471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.318042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.320020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:29.339857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe488a-a511-44b1-ade4-9559aa20207a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.340026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:29.341158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:29.423171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.424079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.424424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.424677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] PATH_INFO: `/v3/projects/05cc12ab75af40bbbb9175c59f089859/users/b1b0b1d906e246c0b7b59dd2cac1c6d9/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.432766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1010/2029] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:29.450704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:29.452968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.456109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:29.469076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05cc12ab75af40bbbb9175c59f089859, user_id=b1b0b1d906e246c0b7b59dd2cac1c6d9, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.476231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.545627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a053a-5586-442b-b1ef-73efc5f78a9b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.551741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.553011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.553011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.553011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.570056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.572689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.585334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1011/2030] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:29.588159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1020/2031] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:29 2022] PUT /identity/v3/projects/05cc12ab75af40bbbb9175c59f089859/users/b1b0b1d906e246c0b7b59dd2cac1c6d9/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:29.592789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.605132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d552-471f-43d0-ad50-c6212162130d service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.675634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.679449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.679818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.680189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.681807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.683547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099a5c1-e3f1-4030-bea4-7629d0477b18 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.695172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1012/2032] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:29.699499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.712551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e5dc-7fb3-482a-a4d8-eb54618cc7a0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.719888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.720994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.721641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.724010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] PATH_INFO: `/v3/projects/05cc12ab75af40bbbb9175c59f089859/users/b1b0b1d906e246c0b7b59dd2cac1c6d9/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.724295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:29.728289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05cc12ab75af40bbbb9175c59f089859, user_id=b1b0b1d906e246c0b7b59dd2cac1c6d9, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.734877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:29.746759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1021/2033] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:29 2022] PUT /identity/v3/projects/05cc12ab75af40bbbb9175c59f089859/users/b1b0b1d906e246c0b7b59dd2cac1c6d9/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:29.763612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.763612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.763612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.803110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] MFA Rules not processed for user `b1b0b1d906e246c0b7b59dd2cac1c6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:29.804567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:29.885139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1013/2034] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:29.897024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:29.902862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:29.980542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bea21e-72fe-47b2-aae9-6738c6137e15 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:29.986002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:29.993788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:29.993788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:29.993788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:29.998397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:29.999973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:30.010379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1022/2035] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:30.270058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:30.271699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:30.288861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:30.308358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da1154-92b8-49d0-9252-6635b59e1cf1 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:30.318063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:30.318747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:30.319032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:30.319310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:30.321591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:30.369731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24eee-6e7f-4e01-a4ce-582b2d542159 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:30.374783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:30.375801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:30.376082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:30.376356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] PATH_INFO: `/v3/projects/b1694069a8304c8d80e9ca8721d423f7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:30.377562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1694069a8304c8d80e9ca8721d423f7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:30.379539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:30.394976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:30.396508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:30.409166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1023/2036] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:30.479867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:30.485504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1694069a8304c8d80e9ca8721d423f7', 'request_id': 'req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.531303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1694069a8304c8d80e9ca8721d423f7', 'request_id': 'req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.542293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1694069a8304c8d80e9ca8721d423f7', 'request_id': 'req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.556533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] The token cache is being invalidate because project b1694069a8304c8d80e9ca8721d423f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:30.556829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.557871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.560392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1014/2037] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:30 2022] DELETE /identity/v3/projects/b1694069a8304c8d80e9ca8721d423f7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:30.564506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:30.579797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df312-ef7c-4da0-ab15-6301466c0792 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:30.614881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:30.654780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:30.659397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:30.659508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:30.659779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] PATH_INFO: `/v3/users/1bb84aa5a2b1475baf9107f2ea909078` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:30.663562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb84aa5a2b1475baf9107f2ea909078)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:30.666669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:30.890042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb84aa5a2b1475baf9107f2ea909078'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.909187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb84aa5a2b1475baf9107f2ea909078'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.910793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb84aa5a2b1475baf9107f2ea909078'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.910793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb84aa5a2b1475baf9107f2ea909078'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.980605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f7e97983-74b0-47b0-8889-735a604eb841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb84aa5a2b1475baf9107f2ea909078'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:30.999606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1024/2038] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:30 2022] DELETE /identity/v3/users/1bb84aa5a2b1475baf9107f2ea909078 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:31.289226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:31.456679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:31.458140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:31.474117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac8125-1a02-45bb-b0b5-4bbd3eb06e0d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:31.538037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:31.538940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:31.539232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:31.539512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] PATH_INFO: `/v3/projects/694b09bb2da54fe29eb9498105f6839e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:31.541238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694b09bb2da54fe29eb9498105f6839e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:31.543204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:31.644430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694b09bb2da54fe29eb9498105f6839e', 'request_id': 'req-da59d76c-8c57-49da-8083-b3bbbb318416'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:31.740439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694b09bb2da54fe29eb9498105f6839e', 'request_id': 'req-da59d76c-8c57-49da-8083-b3bbbb318416'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:31.746665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694b09bb2da54fe29eb9498105f6839e', 'request_id': 'req-da59d76c-8c57-49da-8083-b3bbbb318416'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:31.759829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] The token cache is being invalidate because project 694b09bb2da54fe29eb9498105f6839e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:31.760119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:31.760910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-da59d76c-8c57-49da-8083-b3bbbb318416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:31.762472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1015/2039] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:31 2022] DELETE /identity/v3/projects/694b09bb2da54fe29eb9498105f6839e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:31.768350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:31.768639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:31.768920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:31.791181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:31.792401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:31.797189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:31.869482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1025/2040] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:31.877895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:31.878377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:31.878907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:31.897845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:31.898568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:31.911202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1016/2041] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:31.917303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:31.935093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d74aef-cd10-46e6-818a-03c33fbe960a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:31.942406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:31.949894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:31.965607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1026/2042] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:31.970229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:31.981484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626bca5-ac5d-4216-9582-b06f741b715c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:31.987639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:31.988307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:31.989713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:31.989713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:31.989713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:31.991548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.097022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1017/2043] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:31 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:32.101153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.111710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f3149-e02a-46ef-906e-69e8c9721322 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.161918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:32.178156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.178871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.179155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.179433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.180140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.182252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.298396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1027/2044] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:32 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:32.299654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.310142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7bd21-e513-472d-bcb8-dbe8d5ac5571 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.314847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.315495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.315829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.316193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.316911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.318494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.329652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1018/2045] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:32.333884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.344868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60e3e-cb47-42fe-b14e-c02ca8f7e4da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.349891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.350582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.350866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.351141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.351859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.353731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a616e89-3493-45c0-868f-0e6dede5b45d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.363871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1028/2046] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:32.368181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.378966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d253c-cb8b-4dc2-a704-e1670d32f68d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.384652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.386746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.386746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.386746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] PATH_INFO: `/v3/projects/9848626c53a84a5ab1c8d03b9bdcce62/users/860ecda75a5a464898cd7abc39621186/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.402713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848626c53a84a5ab1c8d03b9bdcce62, user_id=860ecda75a5a464898cd7abc39621186, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.405541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.467969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1019/2047] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:32 2022] PUT /identity/v3/projects/9848626c53a84a5ab1c8d03b9bdcce62/users/860ecda75a5a464898cd7abc39621186/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:32.480797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.481261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.484543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.514218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] MFA Rules not processed for user `860ecda75a5a464898cd7abc39621186`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:32.516438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:32.585466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1029/2048] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:32.598212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.600136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:32.619856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:32.660358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e48f9-8ea6-4701-8ab3-c929c264ecb9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.664051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.664670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.665714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.666153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.674823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.676294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7566311-31c1-48ee-8a48-6995e59def6f service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.686466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1020/2049] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:32.708456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:32.774702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a410f92-0150-4415-a629-0b07c99e71a3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:32.779384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:32.780268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:32.780547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:32.780825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] PATH_INFO: `/v3/users/860ecda75a5a464898cd7abc39621186` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:32.782027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860ecda75a5a464898cd7abc39621186)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:32.784049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:32.948417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860ecda75a5a464898cd7abc39621186'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:32.950534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:32.958019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860ecda75a5a464898cd7abc39621186'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:32.958789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860ecda75a5a464898cd7abc39621186'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:32.959423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860ecda75a5a464898cd7abc39621186'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.049118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860ecda75a5a464898cd7abc39621186'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.070327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1030/2050] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:32 2022] DELETE /identity/v3/users/860ecda75a5a464898cd7abc39621186 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:33.075822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:33.076131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:33.076382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:33.093428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:33.094675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:33.175692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1021/2051] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:33.179771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:33.181334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:33.261180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b293b3-f169-4421-bb30-a46721441bf6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:33.265577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:33.266300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:33.267613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:33.267613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:33.277583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:33.279384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8966b8-ae6a-4122-a7f9-a08c5a537504 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:33.301475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1031/2052] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:33.751826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:33.764617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:33.778431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093d992-e483-4125-bd0e-ad5371666a14 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:33.783599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:33.784503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:33.784790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:33.785072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] PATH_INFO: `/v3/projects/9848626c53a84a5ab1c8d03b9bdcce62` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:33.786323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9848626c53a84a5ab1c8d03b9bdcce62)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:33.788713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:33.919657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9848626c53a84a5ab1c8d03b9bdcce62', 'request_id': 'req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.974136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9848626c53a84a5ab1c8d03b9bdcce62', 'request_id': 'req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.984431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9848626c53a84a5ab1c8d03b9bdcce62', 'request_id': 'req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.997174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] The token cache is being invalidate because project 9848626c53a84a5ab1c8d03b9bdcce62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:33.997515 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.998171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-970f25c2-7a8a-47bf-b0c7-ab0263f597b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:33.999761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1022/2053] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:33 2022] DELETE /identity/v3/projects/9848626c53a84a5ab1c8d03b9bdcce62 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:34.006643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.007152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.007265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.027967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:34.029572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:34.110231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1032/2054] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:34.118898 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.119240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.119461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.142598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:34.142822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:34.152966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1023/2055] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:34.157236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.174667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300bb3-d02a-402f-b7db-15ebf3d33064 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.183593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.184380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.184670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.184949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.185741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:34.186051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.197236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.224875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1033/2056] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:34.229987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.245831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f83ce7-7700-40b5-9926-aa0ea31a3105 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.257501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.258194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.258532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.260141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.260867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.262902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.352757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.360717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1024/2057] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:34 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:34.368540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dfb2c-c5c5-4990-a1c2-580ec66e69bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.462127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:34.468040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.468814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.469106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.469381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.470074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.472291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.553768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1034/2058] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:34 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:34.559277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.569626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cce394-14c0-428d-b525-6d4d0831f8f3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.574957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.576103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.576103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.576103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.576845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.578465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.610316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1025/2059] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:34.616267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.627993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f264a-8dd5-4293-af11-70e33c6b6d9e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.635029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.636114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.636758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.637107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.638014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.641650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d7801-bb42-461f-b244-e1ad55f181a8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.657672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1035/2060] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:34.659007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.664619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:34.671944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65203-8369-4dc4-821c-e68bfb43c4ed admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:34.681335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:34.682473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.682840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.683116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] PATH_INFO: `/v3/projects/9953600e0e164618a2ba8bade9d1f9c5/users/6fc8eb83c4f5442388a5818c85fdade9/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.703139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9953600e0e164618a2ba8bade9d1f9c5, user_id=6fc8eb83c4f5442388a5818c85fdade9, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:34.706314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:34.774938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1026/2061] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:34 2022] PUT /identity/v3/projects/9953600e0e164618a2ba8bade9d1f9c5/users/6fc8eb83c4f5442388a5818c85fdade9/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:34.787111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:34.787583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:34.788024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:34.844074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] MFA Rules not processed for user `6fc8eb83c4f5442388a5818c85fdade9`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:34.845313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:34.919689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1036/2062] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:34.983999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:34.985864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:35.046192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5f0e2-387c-41ed-8167-e079296e4542 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:35.049980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:35.050739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:35.051027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:35.051307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:35.062245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:35.063741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620f27-9a01-4be4-8149-e063a81ee421 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:35.077703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1027/2063] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:40.169250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:40.169750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:40.256040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ad3ac-3b8b-4d04-84e9-6f3c9a10022e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:40.261297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:40.262162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:40.262438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:40.262852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] PATH_INFO: `/v3/users/bb1ebfb16aa54fb5826ec6dacac9233c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:40.264065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1ebfb16aa54fb5826ec6dacac9233c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:40.266020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:40.479450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1ebfb16aa54fb5826ec6dacac9233c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:40.486316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1ebfb16aa54fb5826ec6dacac9233c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:40.487084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1ebfb16aa54fb5826ec6dacac9233c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:40.487705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1ebfb16aa54fb5826ec6dacac9233c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:40.574121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1ebfb16aa54fb5826ec6dacac9233c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:40.580483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:40.598712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1037/2064] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:40 2022] DELETE /identity/v3/users/bb1ebfb16aa54fb5826ec6dacac9233c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:40.603860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26005e5-47b5-4c05-9905-072960945dce None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:40.606921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26005e5-47b5-4c05-9905-072960945dce None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:40.606921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26005e5-47b5-4c05-9905-072960945dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:40.626020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c26005e5-47b5-4c05-9905-072960945dce None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:40.627242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c26005e5-47b5-4c05-9905-072960945dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:40.694153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1028/2065] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:40.701449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:40.702906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:40.763193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399628-7800-437b-a76a-1e1b3eab29a3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:40.766806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:40.767449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:40.767730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:40.768007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:40.777192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:40.778758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:40.788259 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1038/2066] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:41.328174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26005e5-47b5-4c05-9905-072960945dce None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:41.329628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c26005e5-47b5-4c05-9905-072960945dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:41.361691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26005e5-47b5-4c05-9905-072960945dce None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:41.367377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:41.367894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:41.368098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:41.368377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] PATH_INFO: `/v3/projects/cc189f2f94f64882a68608d125f4d248` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:41.370166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc189f2f94f64882a68608d125f4d248)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:41.372125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:41.508763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc189f2f94f64882a68608d125f4d248', 'request_id': 'req-ec94f361-209c-484e-a6ce-d8597b3a2bb8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:41.607760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc189f2f94f64882a68608d125f4d248', 'request_id': 'req-ec94f361-209c-484e-a6ce-d8597b3a2bb8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:41.614654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc189f2f94f64882a68608d125f4d248', 'request_id': 'req-ec94f361-209c-484e-a6ce-d8597b3a2bb8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:41.626685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] The token cache is being invalidate because project cc189f2f94f64882a68608d125f4d248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:41.626880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:41.627561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:41.629241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1029/2067] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:41 2022] DELETE /identity/v3/projects/cc189f2f94f64882a68608d125f4d248 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:41.633345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:41.648202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3e092-0435-4641-90f0-21787114034a service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:41.714836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:41.715698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:41.716004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:41.716279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] PATH_INFO: `/v3/users/b1b0b1d906e246c0b7b59dd2cac1c6d9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:41.717447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b0b1d906e246c0b7b59dd2cac1c6d9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:41.719366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:41.999736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b0b1d906e246c0b7b59dd2cac1c6d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.007001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b0b1d906e246c0b7b59dd2cac1c6d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.007986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b0b1d906e246c0b7b59dd2cac1c6d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.008776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b0b1d906e246c0b7b59dd2cac1c6d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.106256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70c45ec0-85a1-4188-a62e-27749a1181b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b0b1d906e246c0b7b59dd2cac1c6d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.126202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1039/2068] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:41 2022] DELETE /identity/v3/users/b1b0b1d906e246c0b7b59dd2cac1c6d9 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:42.641632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:42.648287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:42.664806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:42.671288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f361-209c-484e-a6ce-d8597b3a2bb8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:42.741930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:42.743694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:42.744225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:42.744569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] PATH_INFO: `/v3/projects/05cc12ab75af40bbbb9175c59f089859` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:42.745969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05cc12ab75af40bbbb9175c59f089859)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:42.747970 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:42.874246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05cc12ab75af40bbbb9175c59f089859', 'request_id': 'req-814a9f25-1dd5-4671-8392-91807fa75c6e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.968725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05cc12ab75af40bbbb9175c59f089859', 'request_id': 'req-814a9f25-1dd5-4671-8392-91807fa75c6e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.979301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05cc12ab75af40bbbb9175c59f089859', 'request_id': 'req-814a9f25-1dd5-4671-8392-91807fa75c6e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.991325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] The token cache is being invalidate because project 05cc12ab75af40bbbb9175c59f089859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:42.991727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.992467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-814a9f25-1dd5-4671-8392-91807fa75c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:42.997154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1030/2069] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:42 2022] DELETE /identity/v3/projects/05cc12ab75af40bbbb9175c59f089859 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:43.003200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.003751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.003842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.018349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:43.026170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:43.027451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:43.106598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1040/2070] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:43.116016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.116456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.116813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.141620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:43.142865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:43.152199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1031/2071] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:43.156319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.170606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ebbf8-2b2b-4a94-9bd4-030467c72689 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.187075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.187075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.187075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.187075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.187407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:43.187407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.187407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.196535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1041/2072] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:43.203018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.217243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6343c1-f9bd-43af-ba59-d970611f2190 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.224140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.225317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.226000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.226456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.227741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.233236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.369796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1032/2073] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:43 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:43.371036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.394752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3528fa-db06-4b67-97f6-9e6807ad5878 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.481927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.482635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.482924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.483195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.483892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.486053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.619550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1042/2074] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:43 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:43.621641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.643763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f751275-0dae-4ffd-b158-10136725cb74 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.660085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.660933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.661220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.661502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.662260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.666043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.691420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.692403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1033/2075] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:43.709415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ad1e-3c1a-4f83-a23d-70b6b515bd3e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.716383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.717553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.718069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.718770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.720089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.722572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb710326-5f56-4b8a-8f7b-d5f0341101b0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.734300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1043/2076] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:43.738812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:43.751086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0c6b-eebc-4a36-b705-9b5228db42e3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:43.760057 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:43.760975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.761436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.761723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] PATH_INFO: `/v3/projects/7b627e5d45a0454bbf9db7f0ef3ca74b/users/39951d10b0a1471eb6844ccab7db5020/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.791078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b627e5d45a0454bbf9db7f0ef3ca74b, user_id=39951d10b0a1471eb6844ccab7db5020, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:43.794985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:43.905145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1034/2077] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:43 2022] PUT /identity/v3/projects/7b627e5d45a0454bbf9db7f0ef3ca74b/users/39951d10b0a1471eb6844ccab7db5020/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:43.916150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844a6e-4685-46e5-a1ad-8ef744c17273 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:43.917039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844a6e-4685-46e5-a1ad-8ef744c17273 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:43.917333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844a6e-4685-46e5-a1ad-8ef744c17273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:43.964199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-86844a6e-4685-46e5-a1ad-8ef744c17273 None None] MFA Rules not processed for user `39951d10b0a1471eb6844ccab7db5020`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:43.967214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-86844a6e-4685-46e5-a1ad-8ef744c17273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:44.075965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1044/2078] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:44.087945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:44.092681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:44.144903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:44.175798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccb55-4cd5-4804-9111-60a5293df3bb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:44.180468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:44.181216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:44.181504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:44.181833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:44.204614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:44.206199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887e88c-f56c-401d-b71a-9dcfa3201d14 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:44.222011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1035/2079] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:47.455954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:47.456477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:47.456477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:47.457737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1045/2080] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:47:47.473156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:47.473704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:47.473809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:47.495584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] MFA Rules not processed for user `f4d6a7f8781c4139a4d927dc47cf1a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:47.496872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:47.566851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1036/2081] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:47.580380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:47.582129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:47.594438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:47.604847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03315b2-4b3a-4995-aa55-67232125d40b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:47.610674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:47.611322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:47.611623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:47.611880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:47.626513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:47.628246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:47.641225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:47.643160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:47.665562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1046/2082] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:47.716795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263f07-ec4d-48ae-b3bd-f5fc271fe119 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:47.721983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:47.723071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:47.723353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:47.723624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] PATH_INFO: `/v3/users/ad7606b56fec4723b733f838ec34a17e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:47.724814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7606b56fec4723b733f838ec34a17e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:47.726778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:47.918728 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7606b56fec4723b733f838ec34a17e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:47.926816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7606b56fec4723b733f838ec34a17e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:47.996986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7606b56fec4723b733f838ec34a17e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:47.997784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7606b56fec4723b733f838ec34a17e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:47.998480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55424e0d-b054-402d-8ab0-b8d5f34c87c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7606b56fec4723b733f838ec34a17e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.008929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:48.018596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1037/2083] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:47 2022] DELETE /identity/v3/users/ad7606b56fec4723b733f838ec34a17e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:48.273715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:48.285750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bebb0-70ad-4d97-9f89-1bc8916aaea8 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:48.344825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:48.345957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.346240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.346511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] PATH_INFO: `/v3/projects/9ceeabe5ea2a4a24b28901f50be5d532` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.347719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ceeabe5ea2a4a24b28901f50be5d532)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:48.349611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:48.451059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ceeabe5ea2a4a24b28901f50be5d532', 'request_id': 'req-13d011e5-47a4-4b6c-9ec0-eb157867fb56'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.505861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ceeabe5ea2a4a24b28901f50be5d532', 'request_id': 'req-13d011e5-47a4-4b6c-9ec0-eb157867fb56'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.513385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ceeabe5ea2a4a24b28901f50be5d532', 'request_id': 'req-13d011e5-47a4-4b6c-9ec0-eb157867fb56'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.523669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] The token cache is being invalidate because project 9ceeabe5ea2a4a24b28901f50be5d532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:48.523953 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.524705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13d011e5-47a4-4b6c-9ec0-eb157867fb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:48.526254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1047/2084] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:48 2022] DELETE /identity/v3/projects/9ceeabe5ea2a4a24b28901f50be5d532 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:48.531241 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.531699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.531810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.549514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:48.550789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:48.611930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1038/2085] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:48.621570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.621865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.622153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.641711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:48.642976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:48.651227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1048/2086] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:48.656221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:48.666696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd88323-b345-4c88-8c33-671c2f411771 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:48.671222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:48.671872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.672151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.672427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.673197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:48.673482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:48.675115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:48.687794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1039/2087] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:48.692275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:48.702181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199366f6-fb8e-4bfa-805e-79bd7bf8f111 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:48.706906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:48.707555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.707839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.708119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.708824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:48.710757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:48.828560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1049/2088] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:48 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:48.832874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:48.843535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a8272-6607-4be6-a59c-96af65535090 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:48.903861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:48.904555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:48.904834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:48.905106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:48.905812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:48.907757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:49.003573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1040/2089] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:48 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:49.007685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:49.018427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158483ef-992d-4bbc-ac42-fedb48915b4d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:49.023208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:49.023878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:49.024232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:49.025103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:49.025901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:49.027546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:49.038828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1050/2090] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:49.043821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:49.054108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfe4b-a03f-4edc-b9b2-a0bb9fd51608 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:49.059022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:49.059703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:49.060454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:49.060454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:49.061037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:49.062686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4a94-c3e9-4db6-b3e3-cf2d379acdc7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:49.073824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1041/2091] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:49.078118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:49.092199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa8964-f882-45f0-b87b-2ebb22bb4740 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:49.098231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:49.099342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:49.100266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:49.100555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] PATH_INFO: `/v3/projects/ed76edca283846f58c6e34e6fbc7599a/users/3cd254f06b234870a6559ac25034ec26/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:49.119594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed76edca283846f58c6e34e6fbc7599a, user_id=3cd254f06b234870a6559ac25034ec26, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:49.122626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:49.186963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1051/2092] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:49 2022] PUT /identity/v3/projects/ed76edca283846f58c6e34e6fbc7599a/users/3cd254f06b234870a6559ac25034ec26/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:49.199945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:49.200234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:49.200513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:49.227358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] MFA Rules not processed for user `3cd254f06b234870a6559ac25034ec26`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:49.228610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:49.274805 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:49.295945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1042/2093] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:49.307775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:49.309489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:49.330030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:49.386938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db470be5-64c8-4ca3-aba6-d3fb8ed8d3bc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:49.392514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:49.393169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:49.393452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:49.393729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:49.401897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:49.403531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3c828-3104-4052-bdb9-b25bb5f42c7f service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:49.412305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1052/2094] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:53.729581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:53.805079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7aa77-7cc6-4937-98bc-ccf3d7d93ac6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:53.811848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:53.812841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:53.813193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:53.813535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] PATH_INFO: `/v3/users/3cd254f06b234870a6559ac25034ec26` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:53.815319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd254f06b234870a6559ac25034ec26)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:53.817434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:54.046096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd254f06b234870a6559ac25034ec26'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.054347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd254f06b234870a6559ac25034ec26'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.121691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd254f06b234870a6559ac25034ec26'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.122635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd254f06b234870a6559ac25034ec26'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.123188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd254f06b234870a6559ac25034ec26'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.141307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1043/2095] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:53 2022] DELETE /identity/v3/users/3cd254f06b234870a6559ac25034ec26 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:54.145868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.146363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.146482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.164586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:54.167013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.215351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:54.217152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.242070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1053/2096] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:54.252443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:54.253849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.289344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf51d-a2f1-4e2e-8cc2-8556aa09e59c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:54.294827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:54.295743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.296024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.296304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] PATH_INFO: `/v3/users/6fc8eb83c4f5442388a5818c85fdade9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.299937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc8eb83c4f5442388a5818c85fdade9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:54.301997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:54.307563 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:54.322594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933eb75-2587-4a7f-b334-600c8a038e1b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:54.327463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:54.328110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.328388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.328662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.338687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:54.340169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b43ea-c948-4376-996d-6a785243cf86 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:54.350299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1054/2097] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:54.592039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc8eb83c4f5442388a5818c85fdade9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.599567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc8eb83c4f5442388a5818c85fdade9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.711219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc8eb83c4f5442388a5818c85fdade9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.712319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc8eb83c4f5442388a5818c85fdade9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.712889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc8eb83c4f5442388a5818c85fdade9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:54.741878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1044/2098] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:54 2022] DELETE /identity/v3/users/6fc8eb83c4f5442388a5818c85fdade9 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:54.748193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.748541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.749938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.769974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:54.771231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.816219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:54.818453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.843444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:54.867904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1055/2099] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:54.884272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:54.887357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:54.919362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35386b-d9d7-4552-bfb2-3abe3917f111 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:54.924016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:54.924869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.925144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.925418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] PATH_INFO: `/v3/projects/ed76edca283846f58c6e34e6fbc7599a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.926776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed76edca283846f58c6e34e6fbc7599a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:54.928856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:54.975590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278badc-f2d0-4818-9e79-aff3e24c5d1e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:54.980274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:54.980993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:54.981277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:54.981585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:54.991837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:54.993602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.004536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1056/2100] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:55.005410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.006763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.020900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731113f4-43b9-49f3-83fb-c72766a34c65 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.027361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.028638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.030943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.030943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] PATH_INFO: `/v3/users/39951d10b0a1471eb6844ccab7db5020` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.030943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39951d10b0a1471eb6844ccab7db5020)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.032753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.046259 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed76edca283846f58c6e34e6fbc7599a', 'request_id': 'req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.128444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed76edca283846f58c6e34e6fbc7599a', 'request_id': 'req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.136451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed76edca283846f58c6e34e6fbc7599a', 'request_id': 'req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.153086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] The token cache is being invalidate because project ed76edca283846f58c6e34e6fbc7599a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:55.153553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.154371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c2b4161e-77b2-4bc1-af8a-0ebddf6b761e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.156380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1045/2101] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:54 2022] DELETE /identity/v3/projects/ed76edca283846f58c6e34e6fbc7599a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:55.160994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3caa0-2e76-4159-8d8e-e4ee78851540 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.161604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3caa0-2e76-4159-8d8e-e4ee78851540 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.163870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3caa0-2e76-4159-8d8e-e4ee78851540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.186472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-75e3caa0-2e76-4159-8d8e-e4ee78851540 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:55.188138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-75e3caa0-2e76-4159-8d8e-e4ee78851540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.199193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:55.264991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1046/2102] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:55.272573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.272863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.273142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.294284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:55.295771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.296055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39951d10b0a1471eb6844ccab7db5020'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.309841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39951d10b0a1471eb6844ccab7db5020'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.314109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39951d10b0a1471eb6844ccab7db5020'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.314109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39951d10b0a1471eb6844ccab7db5020'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.314408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1047/2103] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:55.314408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.322608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950f55e-6d49-4c11-8802-38aedd63fed6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.327339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.328000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.328279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.328554 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.329288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:55.329593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.331353 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.346584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1048/2104] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:47:55.348804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.350629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.369609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cecbd-7b49-45d1-a62b-e9f48665b9df admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.375154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.375938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.376302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.376678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.378528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.380643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.387975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93229a69-6162-46cd-8d80-337ab8e68d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39951d10b0a1471eb6844ccab7db5020'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.412821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1057/2105] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:47:55 2022] DELETE /identity/v3/users/39951d10b0a1471eb6844ccab7db5020 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:55.418460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.418460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.418696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.437706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:55.439159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.492049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1049/2106] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:55.496547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.515681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:55.529492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1058/2107] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:55.542951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.544741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.587540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a540c9-c7d9-4bff-9035-cff7afe5b4b3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.593147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.593838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.594120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.594401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.595143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.597949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.619412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5971e88-d8c2-4a2e-bb11-36a19bf23b3a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.677010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.677669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.677951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.678231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.688928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1050/2108] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:55.689795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.690022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.691740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.692779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.702320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1059/2109] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:47:55.703197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.705423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c3824-1f3c-43f8-8c7f-3c9bb46539cf admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.711225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.712818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.713183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.713467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] PATH_INFO: `/v3/projects/9953600e0e164618a2ba8bade9d1f9c5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.713702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb94d-30e3-4eca-82e6-bb70ae554b6a service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.715034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9953600e0e164618a2ba8bade9d1f9c5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.716978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.718803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.719530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.731930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.731930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.731930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.731930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.735392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1060/2110] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:55.739291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.750536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bb030-14fa-48f2-8166-baea5672122e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.755483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.756164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.756450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.756912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.757426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.759197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.769967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1061/2111] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:55.774020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:55.784796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540164c3-3b5f-42c8-8311-2bc68c7776ee admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:55.792714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:55.793613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.793897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.794176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] PATH_INFO: `/v3/projects/008335b6af0642bd8e78fc73acdb5c05/users/df4f8557fcb14baa847a2fbb2db6d7ae/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.814324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=008335b6af0642bd8e78fc73acdb5c05, user_id=df4f8557fcb14baa847a2fbb2db6d7ae, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:55.817028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34affe2f-1aa0-46cd-be30-2f8bfda0c40d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:55.828684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9953600e0e164618a2ba8bade9d1f9c5', 'request_id': 'req-53185f4b-96d4-4f77-a559-4891eb8d76c3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.880788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1062/2112] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:55 2022] PUT /identity/v3/projects/008335b6af0642bd8e78fc73acdb5c05/users/df4f8557fcb14baa847a2fbb2db6d7ae/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:55.893508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:55.893866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:55.894154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:55.917406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9953600e0e164618a2ba8bade9d1f9c5', 'request_id': 'req-53185f4b-96d4-4f77-a559-4891eb8d76c3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.918944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:55.927601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9953600e0e164618a2ba8bade9d1f9c5', 'request_id': 'req-53185f4b-96d4-4f77-a559-4891eb8d76c3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.932226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] MFA Rules not processed for user `df4f8557fcb14baa847a2fbb2db6d7ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:55.933419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:55.948101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] The token cache is being invalidate because project 9953600e0e164618a2ba8bade9d1f9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:55.948441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.949123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:55.951105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1051/2113] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:55 2022] DELETE /identity/v3/projects/9953600e0e164618a2ba8bade9d1f9c5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:56.018911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1063/2114] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:56.020251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.024005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.129106 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185f4b-96d4-4f77-a559-4891eb8d76c3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.196792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.197455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.197736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.198014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.198744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.200688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.247855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.247855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.256021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:56.262935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e5a68-87fc-42c1-8aa2-ec40f4393e77 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.268954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.270365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.270878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.271386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] PATH_INFO: `/v3/projects/7b627e5d45a0454bbf9db7f0ef3ca74b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.272659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b627e5d45a0454bbf9db7f0ef3ca74b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.274640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.304803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1052/2115] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/projects => generated 379 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:56.306199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.368306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7a5b9-e215-4836-a95e-652229a0826c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.373123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.373820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.374081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.374413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.375317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.377242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.404542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b627e5d45a0454bbf9db7f0ef3ca74b', 'request_id': 'req-73d86a07-e118-4b14-ad49-81a478ce8fb6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:56.459623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b627e5d45a0454bbf9db7f0ef3ca74b', 'request_id': 'req-73d86a07-e118-4b14-ad49-81a478ce8fb6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:56.469729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1053/2116] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:56.469923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b627e5d45a0454bbf9db7f0ef3ca74b', 'request_id': 'req-73d86a07-e118-4b14-ad49-81a478ce8fb6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:56.471429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.479614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] The token cache is being invalidate because project 7b627e5d45a0454bbf9db7f0ef3ca74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:47:56.479907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:56.480741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73d86a07-e118-4b14-ad49-81a478ce8fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:47:56.482443 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1064/2117] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:47:56 2022] DELETE /identity/v3/projects/7b627e5d45a0454bbf9db7f0ef3ca74b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:56.482590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957c05a-d5c4-4dc8-84d6-4c51fb47ccbe admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.487295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76201b4-3ac5-434d-9bf1-f0af2933cf19 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.487576 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76201b4-3ac5-434d-9bf1-f0af2933cf19 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.487877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76201b4-3ac5-434d-9bf1-f0af2933cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.506185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f76201b4-3ac5-434d-9bf1-f0af2933cf19 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:56.507430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f76201b4-3ac5-434d-9bf1-f0af2933cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.543925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.544589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.544869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.545143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.545872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.547513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.560294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1054/2118] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:56.567861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.569703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.581120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1065/2119] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:56.582807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:56.591241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.591773 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.592229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.592833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d7cb-01e7-4257-822b-03010747d6a0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.598291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.599215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.599542 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.599847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] PATH_INFO: `/v3/projects/52849db8ba404dbea18a007483c2e235/users/c5fd764aa18d4f5187dcd15663ae254a/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.612756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:56.619997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.620110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52849db8ba404dbea18a007483c2e235, user_id=c5fd764aa18d4f5187dcd15663ae254a, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.620110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.623108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1066/2120] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:56.627220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.636503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475e409-d268-4844-9049-9b5f3f525427 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.641860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.642515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.642850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.646781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.646781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:47:56.646781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.646781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.658287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1067/2121] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:47:56.662366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.673478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864b9f2-459b-4f0b-8658-d85697725dc1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.679197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.679855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.687027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.687027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.687027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.687027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.738585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1055/2122] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:56 2022] PUT /identity/v3/projects/52849db8ba404dbea18a007483c2e235/users/c5fd764aa18d4f5187dcd15663ae254a/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:47:56.739647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.799830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43943eb6-0a20-4858-9bb5-8acd6b72ac4f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.804906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.805680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.805942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.806218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.806975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.808574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.813298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1068/2123] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:47:56.814302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.819562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1056/2124] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:56.824362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:56.834038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6766b-850a-4488-91e2-107d90fb5d7e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.864835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:56.881649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db3402-151d-433b-a6af-cc4e3f0d6833 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:56.886900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.887566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.887852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.888126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.888951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.890885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.891687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:56.892576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.892856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.893129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] PATH_INFO: `/v3/projects/52849db8ba404dbea18a007483c2e235/users/c5fd764aa18d4f5187dcd15663ae254a/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.895161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52849db8ba404dbea18a007483c2e235, user_id=c5fd764aa18d4f5187dcd15663ae254a, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:56.897800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568edb79-4d54-4630-944e-d322b2e0ac28 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:56.910177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1057/2125] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:56 2022] PUT /identity/v3/projects/52849db8ba404dbea18a007483c2e235/users/c5fd764aa18d4f5187dcd15663ae254a/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:56.921317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:56.921549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:56.921954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:56.951723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] MFA Rules not processed for user `c5fd764aa18d4f5187dcd15663ae254a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:56.952945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:56.989004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1069/2126] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:47:56.990724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.001742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b63be-c814-4691-ae96-c3b028fb916a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.022561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1058/2127] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:47:57.034359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.037206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:57.038034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:57.061328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.061993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.062274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.062563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.063292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.064885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.078670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1070/2128] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:47:57.079933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.089982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0c7c4-6353-4d5d-bfdb-e58c06974ea1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.094811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.095721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.096000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.096270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.096404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26507d-9116-4ae3-ba8e-e0ee21b0283d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.097038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.098786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.100286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.100937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.101213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.101507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.110215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1071/2129] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:47:57.111685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.113177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca88f58-f084-4ef2-adb4-39261a5bf9d6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.114317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.124429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62541d-f3f0-446c-9b6e-93eee80aac58 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.124935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1059/2130] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:57.129194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.130065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.130342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.130659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] PATH_INFO: `/v3/projects/c9a22002be7b46d5b4038424c68c4ba7/users/07c25d060b9e4a8e91c3dbc40e0e5136/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.146325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a22002be7b46d5b4038424c68c4ba7, user_id=07c25d060b9e4a8e91c3dbc40e0e5136, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.148997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.277454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1072/2131] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:47:57 2022] PUT /identity/v3/projects/c9a22002be7b46d5b4038424c68c4ba7/users/07c25d060b9e4a8e91c3dbc40e0e5136/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:47:57.288553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.288841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.289117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.317239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] MFA Rules not processed for user `07c25d060b9e4a8e91c3dbc40e0e5136`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:47:57.318429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:57.359836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.392007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1060/2132] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:47:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:47:57.403962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:47:57.405356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:57.427389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde6fdb-0372-4581-84b0-742998fa52a0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.432790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.433445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.433722 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.434012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.435804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:47:57.439381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:57.473087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4352-72e8-4fff-a880-0cae285c9409 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:47:57.476776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:47:57.477474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:47:57.477791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:47:57.478072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:47:57.485972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:47:57.489199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.490668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.499161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:47:57.500530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0ec9-c931-422a-80b8-df533925dc72 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:47:57.500892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1061/2133] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:47:57.511926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1073/2134] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:47:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:48:07.516911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=14 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:07.693793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:17.616374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:17.692845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f0234-05e0-49f2-8754-60f62b80901d service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:17.697664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:17.698611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:17.698920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:17.699163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] PATH_INFO: `/v3/users/df4f8557fcb14baa847a2fbb2db6d7ae` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:17.700384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4f8557fcb14baa847a2fbb2db6d7ae)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:17.702420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:17.916444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4f8557fcb14baa847a2fbb2db6d7ae'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:17.925706 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4f8557fcb14baa847a2fbb2db6d7ae'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:18.031107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4f8557fcb14baa847a2fbb2db6d7ae'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:18.031968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4f8557fcb14baa847a2fbb2db6d7ae'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:18.032583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4f8557fcb14baa847a2fbb2db6d7ae'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:18.056754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1062/2135] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:17 2022] DELETE /identity/v3/users/df4f8557fcb14baa847a2fbb2db6d7ae => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:18.059807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:18.060384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:18.060777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:18.082709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:18.084080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:18.162787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1074/2136] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:18.172909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:18.174343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:18.238431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c036a22-8f06-45ac-9f66-0fca0795a040 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:18.242304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:18.242985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:18.243269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:18.243543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:18.253869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:18.255275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:18.267339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1063/2137] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:18.792969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:18.794616 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:18.809486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84e411-4e50-44bf-9fb2-22f5226f7e0c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:18.815159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:18.815939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:18.816231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:18.816516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] PATH_INFO: `/v3/projects/008335b6af0642bd8e78fc73acdb5c05` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:18.817880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008335b6af0642bd8e78fc73acdb5c05)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:18.820148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:18.941049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008335b6af0642bd8e78fc73acdb5c05', 'request_id': 'req-9eca0f36-111f-4f72-b8e0-8a8086fb0702'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.022336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008335b6af0642bd8e78fc73acdb5c05', 'request_id': 'req-9eca0f36-111f-4f72-b8e0-8a8086fb0702'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.028778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008335b6af0642bd8e78fc73acdb5c05', 'request_id': 'req-9eca0f36-111f-4f72-b8e0-8a8086fb0702'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.039949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] The token cache is being invalidate because project 008335b6af0642bd8e78fc73acdb5c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:19.040348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.040876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.046305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1075/2138] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:18 2022] DELETE /identity/v3/projects/008335b6af0642bd8e78fc73acdb5c05 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:19.047614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:19.058579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e3a1-8d8a-4c73-abda-1a719c27bd17 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:19.079346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:19.132843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:19.133990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:19.134272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:19.134565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] PATH_INFO: `/v3/users/c5fd764aa18d4f5187dcd15663ae254a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:19.135782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fd764aa18d4f5187dcd15663ae254a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:19.138945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:19.389326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fd764aa18d4f5187dcd15663ae254a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.396355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fd764aa18d4f5187dcd15663ae254a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.481100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fd764aa18d4f5187dcd15663ae254a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.481734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fd764aa18d4f5187dcd15663ae254a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.482331 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-401676a9-0cfc-40ac-944c-0e7c2177763a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fd764aa18d4f5187dcd15663ae254a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:19.500665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1064/2139] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:19 2022] DELETE /identity/v3/users/c5fd764aa18d4f5187dcd15663ae254a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:19.917200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:19.918577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:19.933704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca0f36-111f-4f72-b8e0-8a8086fb0702 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:19.992622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:19.993478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:19.993763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:19.994042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] PATH_INFO: `/v3/projects/52849db8ba404dbea18a007483c2e235` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:19.995251 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52849db8ba404dbea18a007483c2e235)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:19.997907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.129093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:20.146753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52849db8ba404dbea18a007483c2e235', 'request_id': 'req-e3641d09-3783-4a46-a39a-7438fb846642'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:20.213206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52849db8ba404dbea18a007483c2e235', 'request_id': 'req-e3641d09-3783-4a46-a39a-7438fb846642'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:20.220790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52849db8ba404dbea18a007483c2e235', 'request_id': 'req-e3641d09-3783-4a46-a39a-7438fb846642'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:20.231981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] The token cache is being invalidate because project 52849db8ba404dbea18a007483c2e235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:20.232321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:20.232951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3641d09-3783-4a46-a39a-7438fb846642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:20.239356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1076/2140] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:19 2022] DELETE /identity/v3/projects/52849db8ba404dbea18a007483c2e235 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:20.241308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.241784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.241882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.259844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:20.262956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:20.333718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1065/2141] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:20.343310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.343869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.344483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.373510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:20.374765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:20.383035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1077/2142] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:20.387451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.398801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7badc-d4b9-4cfb-ab65-93052c86bf43 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.404590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.405264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.405547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.405824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.406616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:20.406893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.408512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.421943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1066/2143] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:20.426265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.436669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db202df-f4a5-4f0d-8373-5ad835005de5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.441421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.442143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.442422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.442737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.443425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.445332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.529869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1078/2144] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:20 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:20.534236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.544975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f323-0d0e-4114-a551-be53b48b20ac admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.571237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:20.606580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.607407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.607694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.607973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.608707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.610679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.704808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1067/2145] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:20 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:20.708634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.719723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefb7d8-fd6d-476c-a476-64c14a4f0e87 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.724670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.725366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.725647 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.725923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.726688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.728324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.740336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1079/2146] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:20.744447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.754943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802376c0-68ae-4849-a8e6-2fc9e664b22d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.762095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.762798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.763083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.763360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.765232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.765704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87c39-832f-4388-a356-48b046f7b343 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.776495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1068/2147] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:20.781727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:20.793174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149be6d-4300-4e2e-80db-2a2fc9d3bdc8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:20.798831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:20.799404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.799685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.799963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] PATH_INFO: `/v3/projects/303a824a24db445cb62d0714e4b572f5/users/d5675da90d8846469de427ac6a81b540/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.816745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] RBAC: Authorizing `identity:create_grant(project_id=303a824a24db445cb62d0714e4b572f5, user_id=d5675da90d8846469de427ac6a81b540, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:20.819479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:20.928461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1080/2148] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:20 2022] PUT /identity/v3/projects/303a824a24db445cb62d0714e4b572f5/users/d5675da90d8846469de427ac6a81b540/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:20.940071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:20.940443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:20.940731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:20.968492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] MFA Rules not processed for user `d5675da90d8846469de427ac6a81b540`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:20.969611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:21.047300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1069/2149] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:21.051794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.116512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138545c-a139-4aac-b20a-ab1572d1225a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.122227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.122927 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.123233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.123512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.124261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.126189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.226153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1081/2150] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:21 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:21.233254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.244273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf888ec-26ea-400b-9b57-a6501d53cfe4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.304573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.305272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.305552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.305827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.306572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.309374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.393948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1070/2151] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:21 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:21.395067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.406611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df2d95-948d-4d41-9a08-347adc6795b0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.411392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.412067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.412438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.412631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.413392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.415210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.428067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1082/2152] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:21.432367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.442827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a14c90-d5c0-4eda-8450-bea736b1c245 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.447525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.448506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.449010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.449291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] PATH_INFO: `/v3/projects/2aeca6ede4934c9385635ef5f28f6c10/users/d28ad3f453884d40b5e28d835386df49/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.465500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeca6ede4934c9385635ef5f28f6c10, user_id=d28ad3f453884d40b5e28d835386df49, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.468228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.564096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1071/2153] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:21 2022] PUT /identity/v3/projects/2aeca6ede4934c9385635ef5f28f6c10/users/d28ad3f453884d40b5e28d835386df49/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:21.568183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.579748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2db609-cdee-4a9b-a06a-9bdcc8bf53f2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.601309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:21.642488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.643213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.643503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.643782 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.644519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.646139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62401d-5ea1-48ee-9210-2d529a06a0f0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.662338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1083/2154] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:21.666475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.672115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:21.678231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddcd3e-c842-45f1-8916-54026412e698 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.683236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.684131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.684422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.684702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] PATH_INFO: `/v3/projects/2aeca6ede4934c9385635ef5f28f6c10/users/d28ad3f453884d40b5e28d835386df49/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.686998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeca6ede4934c9385635ef5f28f6c10, user_id=d28ad3f453884d40b5e28d835386df49, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.689755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.701680 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1072/2155] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:21 2022] PUT /identity/v3/projects/2aeca6ede4934c9385635ef5f28f6c10/users/d28ad3f453884d40b5e28d835386df49/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:21.712302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.712591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.712871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.743255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] MFA Rules not processed for user `d28ad3f453884d40b5e28d835386df49`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:21.744467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:21.822301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1084/2156] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:21.834183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:21.836094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:21.897788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933a0-c94f-4783-830e-f9038282dda0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:21.902088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:21.902770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:21.903115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:21.903390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:21.913474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:21.915082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:21.926019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1073/2157] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:22.035099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:22.036811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:22.049804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6eb4-f20f-4736-a02c-9d0e2ccf28d9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:22.053490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:22.054151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:22.054434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:22.054740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:22.115947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:22.117317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32df34-1c43-4888-bb1e-780cc8e0e558 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:22.128012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1085/2158] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:23.502402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:23.503908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:23.584692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9226-6820-4238-94f0-cdac1cd7e172 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:23.589437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:23.590338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:23.590653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:23.590933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] PATH_INFO: `/v3/users/7c88f8bcf566470db866540c2bbb12ce` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:23.592173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c88f8bcf566470db866540c2bbb12ce)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:23.594131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:23.815097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c88f8bcf566470db866540c2bbb12ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:23.823414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c88f8bcf566470db866540c2bbb12ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:23.915101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c88f8bcf566470db866540c2bbb12ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:23.927010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c88f8bcf566470db866540c2bbb12ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:23.927010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c88f8bcf566470db866540c2bbb12ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:23.946827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1074/2159] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:23 2022] DELETE /identity/v3/users/7c88f8bcf566470db866540c2bbb12ce => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:23.951843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:23.959353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:23.959353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:23.975817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:23.977078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:24.031388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:24.054990 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1086/2160] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:24.066969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:24.068448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:24.098817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:24.162232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a3a61-1ce7-4f68-9527-e36d57012e4b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:24.165951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:24.166659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:24.167134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:24.168055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:24.178075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:24.179509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e3eb9-a16e-4711-a7ec-9e01e960c6ed service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:24.197501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1075/2161] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:24.664084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:24.666206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:24.691357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bb4ae-625f-46cd-be55-305dd592e308 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:24.702774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:24.703950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:24.705321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:24.705321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] PATH_INFO: `/v3/projects/87eb037c16464a9887edc080790f29eb` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:24.706048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87eb037c16464a9887edc080790f29eb)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:24.708082 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:24.816221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87eb037c16464a9887edc080790f29eb', 'request_id': 'req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:24.918930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87eb037c16464a9887edc080790f29eb', 'request_id': 'req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:24.927162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87eb037c16464a9887edc080790f29eb', 'request_id': 'req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:24.945433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] The token cache is being invalidate because project 87eb037c16464a9887edc080790f29eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:24.945884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:24.947183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ba8c2e2-bda7-4638-b534-3d4cc0354c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:24.950316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1087/2162] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:24 2022] DELETE /identity/v3/projects/87eb037c16464a9887edc080790f29eb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:24.962893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:24.963374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:24.963466 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:24.983813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:24.990746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:25.057609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1076/2163] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:25.066249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.066863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.066863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.085631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:25.086869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:25.096295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1088/2164] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:25.100613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.112051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c20d71-f857-4da7-896a-98f5ec7ff203 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.117207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.118143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.118455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.118774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.119620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:25.119931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.122142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.134664 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1077/2165] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:25.141427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.156712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbff845-e0c6-4011-a99b-faeceef5c390 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.160910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.161620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.164815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.164815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.164815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.165318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.256973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1089/2166] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:25 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:25.261990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.279802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0e68-836c-40d3-9631-dafe8be7740e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.372985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.373670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.373952 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.374229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.374996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.376964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.483407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1078/2167] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:25 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:25.484570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.495672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e42b-c77a-4635-92ed-12ce7a5c4e6d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.501062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.501755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.502025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.506779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.506779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.506779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.517046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1090/2168] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:25.521229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.532437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c98dc-5a2b-41f2-b325-b3f647028247 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.537152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.537823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.538111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.538385 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.539251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.540795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e5e6-cf2f-443b-83e0-0db02175a556 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.553726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1079/2169] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:25.558895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.569344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4c850-e68b-4251-96e8-074d357f46f8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.575031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.575941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.576218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.576484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] PATH_INFO: `/v3/projects/24d5a2477eb74d5da5911b37f9031110/users/26ccc2d61c7045a8abba95e7cd002a0a/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.597684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d5a2477eb74d5da5911b37f9031110, user_id=26ccc2d61c7045a8abba95e7cd002a0a, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.597684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.654225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1091/2170] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:25 2022] PUT /identity/v3/projects/24d5a2477eb74d5da5911b37f9031110/users/26ccc2d61c7045a8abba95e7cd002a0a/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:25.663838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1788-a3da-450e-a5ac-9b36851d5361 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.664435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1788-a3da-450e-a5ac-9b36851d5361 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.664783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1788-a3da-450e-a5ac-9b36851d5361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.699410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b0da1788-a3da-450e-a5ac-9b36851d5361 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:25.700883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b0da1788-a3da-450e-a5ac-9b36851d5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:25.716215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:25.814099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1080/2171] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:25.830048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:25.832706 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:25.902750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e57af-195d-47e0-9118-b590f0457970 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:25.909101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:25.909759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:25.910043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:25.910332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:25.920673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:25.922159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:25.934132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1092/2172] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:27.265632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:27.266123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:27.266339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:27.294533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] MFA Rules not processed for user `26ccc2d61c7045a8abba95e7cd002a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:27.296210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:27.364824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1081/2173] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:27.376853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:27.377381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:27.378537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:27.378670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:27.388860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:27.416446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5fca3-a5d3-43c1-a5ab-f61775bfc24c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:27.416446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:27.416446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:27.416446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:27.417785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] PATH_INFO: `/v3/users/d5675da90d8846469de427ac6a81b540` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:27.417785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5675da90d8846469de427ac6a81b540)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:27.417785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:27.441000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822939f2-d16b-485b-8d1b-1d88203bc7e9 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:27.445523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:27.446170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:27.446452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:27.446756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:27.456091 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:27.457677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcd774-ba8d-43b6-98ea-a4d6e94510dd service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:27.468190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1093/2174] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:27.640300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5675da90d8846469de427ac6a81b540'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:27.647744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5675da90d8846469de427ac6a81b540'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:27.795818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5675da90d8846469de427ac6a81b540'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:27.796445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5675da90d8846469de427ac6a81b540'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:27.797051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5675da90d8846469de427ac6a81b540'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:27.815440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1082/2175] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:27 2022] DELETE /identity/v3/users/d5675da90d8846469de427ac6a81b540 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:27.820045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:27.820476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:27.820821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:27.851900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:27.856655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:27.920839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1094/2176] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:27.930402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:27.931933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:27.963384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:27.999547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217534c-88f1-4d23-8933-3896cb8fe052 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:28.003130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:28.003770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:28.004048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:28.004320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:28.014858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:28.016479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:28.028929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1083/2177] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:28.577150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:28.578741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:28.591454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa4954-cf88-4f65-aa57-2fdc2dd45938 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:28.596921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:28.597762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:28.598037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:28.598486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] PATH_INFO: `/v3/projects/303a824a24db445cb62d0714e4b572f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:28.604044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303a824a24db445cb62d0714e4b572f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:28.604044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:28.748214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303a824a24db445cb62d0714e4b572f5', 'request_id': 'req-589a17ab-2485-4868-a626-843b8658f1c1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:28.800677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303a824a24db445cb62d0714e4b572f5', 'request_id': 'req-589a17ab-2485-4868-a626-843b8658f1c1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:28.809432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303a824a24db445cb62d0714e4b572f5', 'request_id': 'req-589a17ab-2485-4868-a626-843b8658f1c1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:28.821607 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] The token cache is being invalidate because project 303a824a24db445cb62d0714e4b572f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:28.822015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:28.824322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-589a17ab-2485-4868-a626-843b8658f1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:28.826471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1095/2178] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:28 2022] DELETE /identity/v3/projects/303a824a24db445cb62d0714e4b572f5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:28.830504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:28.840319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd874df-ec51-4916-bf3d-b3120a115806 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:28.900550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:28.901786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:28.902088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:28.902486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] PATH_INFO: `/v3/users/d28ad3f453884d40b5e28d835386df49` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:28.903773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28ad3f453884d40b5e28d835386df49)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:28.906242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:29.304314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28ad3f453884d40b5e28d835386df49'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:29.315661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28ad3f453884d40b5e28d835386df49'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:29.407626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:29.407961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:29.408248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:29.427649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] MFA Rules not processed for user `26ccc2d61c7045a8abba95e7cd002a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:29.429011 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:29.461273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:29.476620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28ad3f453884d40b5e28d835386df49'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:29.479323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28ad3f453884d40b5e28d835386df49'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:29.479323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-668ca260-2a01-4e95-bb42-b2a89e341341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28ad3f453884d40b5e28d835386df49'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:29.498996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1096/2179] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:29.524512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1084/2180] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:28 2022] DELETE /identity/v3/users/d28ad3f453884d40b5e28d835386df49 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:29.723721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:29.725412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:29.757363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed0825-550d-4723-b8e5-1f18b4d8372f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:29.828555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:29.829465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:29.829755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:29.830038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] PATH_INFO: `/v3/users/07c25d060b9e4a8e91c3dbc40e0e5136` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:29.831877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c25d060b9e4a8e91c3dbc40e0e5136)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:29.833873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:30.107523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c25d060b9e4a8e91c3dbc40e0e5136'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.115403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c25d060b9e4a8e91c3dbc40e0e5136'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.116705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c25d060b9e4a8e91c3dbc40e0e5136'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.117357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c25d060b9e4a8e91c3dbc40e0e5136'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.213413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c25d060b9e4a8e91c3dbc40e0e5136'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.233163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1097/2181] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:29 2022] DELETE /identity/v3/users/07c25d060b9e4a8e91c3dbc40e0e5136 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:30.238066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.238505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.238505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.248959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:30.250861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.259524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:30.260715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.326637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:30.336743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49b6aa-777a-4c29-99d9-5955f7221a94 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:30.342683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:30.343634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.343906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.344184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] PATH_INFO: `/v3/projects/2aeca6ede4934c9385635ef5f28f6c10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.345633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aeca6ede4934c9385635ef5f28f6c10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:30.347711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:30.355918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1085/2182] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:30.362866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:30.364289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.423429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a97cfa-9f99-4693-aa87-c487f74467b5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:30.426974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:30.427637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.427921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.428200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.438628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:30.440191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e3b98-ca10-494c-8fcf-27207f936e24 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:30.450657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1086/2183] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:30.532316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aeca6ede4934c9385635ef5f28f6c10', 'request_id': 'req-612b60f0-4a1f-42af-99ae-95998d5e6e82'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.618904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aeca6ede4934c9385635ef5f28f6c10', 'request_id': 'req-612b60f0-4a1f-42af-99ae-95998d5e6e82'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.627476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aeca6ede4934c9385635ef5f28f6c10', 'request_id': 'req-612b60f0-4a1f-42af-99ae-95998d5e6e82'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.642790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] The token cache is being invalidate because project 2aeca6ede4934c9385635ef5f28f6c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:30.643004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.643710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:30.650047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1098/2184] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:30 2022] DELETE /identity/v3/projects/2aeca6ede4934c9385635ef5f28f6c10 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:30.651838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba6e6-ad01-4fad-9cf9-64dc9deee675 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.652904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba6e6-ad01-4fad-9cf9-64dc9deee675 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.653193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba6e6-ad01-4fad-9cf9-64dc9deee675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.678821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-928ba6e6-ad01-4fad-9cf9-64dc9deee675 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:30.680115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-928ba6e6-ad01-4fad-9cf9-64dc9deee675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.691146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:30.695302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:30.696243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.762724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1087/2185] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:30.771682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.772159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.772916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.793408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:30.794754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:30.801998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b60f0-4a1f-42af-99ae-95998d5e6e82 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:30.803629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1088/2186] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:30.807813 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:30.813270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:30.813270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.813270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.813270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] PATH_INFO: `/v3/users/0e3c4968462c4cfe9824e0947813f635` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.815217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3c4968462c4cfe9824e0947813f635)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:30.817697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:30.818328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f063-482a-40b4-b7b5-d877c2cdc737 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:30.823749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:30.825985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.825985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.825985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.826785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:30.827352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:30.828968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:30.852823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1089/2187] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:48:30.863228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:30.874359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563867-78f8-47c3-9629-38688f30d16d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:30.883183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:30.884547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:30.884904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:30.885418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:30.887548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:30.895310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.046632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1090/2188] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:30 2022] POST /identity/v3/projects => generated 401 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:48:31.048411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.049860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3c4968462c4cfe9824e0947813f635'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.056812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3c4968462c4cfe9824e0947813f635'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.057490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3c4968462c4cfe9824e0947813f635'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.058837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3c4968462c4cfe9824e0947813f635'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.113710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49c980-f023-40e6-b71f-963d645dfc5d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.118833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.119508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.119789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.120062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.121404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.123553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f440f-4a3a-41a6-afa8-bb91c9e9b9ae admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.150360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3c4968462c4cfe9824e0947813f635'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.169527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1099/2189] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:30 2022] DELETE /identity/v3/users/0e3c4968462c4cfe9824e0947813f635 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:31.173686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.178407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.240096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1091/2190] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/users => generated 301 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:31.240096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3f50-f712-4c08-862a-4ff441446a62 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.240096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3f50-f712-4c08-862a-4ff441446a62 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.240096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3f50-f712-4c08-862a-4ff441446a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.240861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9a4bd-14ce-438b-ae87-30a5cd55a158 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.244140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.245003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.245287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.245568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] PATH_INFO: `/v3/projects/c9a22002be7b46d5b4038424c68c4ba7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.247236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a22002be7b46d5b4038424c68c4ba7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.250651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.258326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-237f3f50-f712-4c08-862a-4ff441446a62 None None] MFA Rules not processed for user `a2cb1079722947dfa37946c03b7297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:31.259567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-237f3f50-f712-4c08-862a-4ff441446a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.316646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1092/2191] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:31.317961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.318231 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.318510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.336001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:31.337233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.354457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a22002be7b46d5b4038424c68c4ba7', 'request_id': 'req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.405948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1093/2192] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:31.406942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.410160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.423560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55729160-041f-4f18-9c00-6eaaa5593d5f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.428200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.429507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.429807 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.430504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.431279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.432889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.443886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1094/2193] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:31.444727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.446988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.458425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a22002be7b46d5b4038424c68c4ba7', 'request_id': 'req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.466054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a22002be7b46d5b4038424c68c4ba7', 'request_id': 'req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.477843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] The token cache is being invalidate because project c9a22002be7b46d5b4038424c68c4ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:31.478220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.478983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:31.481044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1100/2194] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:31 2022] DELETE /identity/v3/projects/c9a22002be7b46d5b4038424c68c4ba7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:31.481660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.483398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.497957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2a0eb-380e-4289-b6b8-bc7414582ce5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.503053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.503879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.504156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.504508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.505551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.518445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd29153-d665-4e60-af7c-1d68869cc91b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.525136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504b56-de19-4bbc-927f-76a1e7adf3f5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.530388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.531544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.531983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.533182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.534135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1101/2195] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:31.535594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de645-8554-4264-aa97-7d9ad586cbe7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.535887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de645-8554-4264-aa97-7d9ad586cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.536170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de645-8554-4264-aa97-7d9ad586cbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.536764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.551427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.553072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.556250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-858de645-8554-4264-aa97-7d9ad586cbe7 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:31.557448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-858de645-8554-4264-aa97-7d9ad586cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.567141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1095/2196] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:31.567141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.573137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1102/2197] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:31.578319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1898a-2258-4496-b499-5ed6b15591f1 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.582201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.582698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.583124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.586317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.587939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.588461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.588858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] PATH_INFO: `/v3/projects/6b1f4202d97044efb0af8cbd33213911/users/82163223dae94b158e73c2b26c8e6c81/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.607521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:31.607975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:31.609090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.614579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1f4202d97044efb0af8cbd33213911, user_id=82163223dae94b158e73c2b26c8e6c81, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.617745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3539081-4782-4f86-b79a-13f81462ab88 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.619458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1103/2198] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:31.624544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.636993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7c306-66ed-4570-937d-f913c915a89b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.643158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.644441 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.644859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.645640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.645971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:31.646272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.648713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.665745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1104/2199] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:48:31.667049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.694120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5134f6-2cf7-4a2c-9106-3f0d9dfa9e27 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.708864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1096/2200] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:31 2022] PUT /identity/v3/projects/6b1f4202d97044efb0af8cbd33213911/users/82163223dae94b158e73c2b26c8e6c81/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:31.710391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.710759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.711043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.750721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] MFA Rules not processed for user `82163223dae94b158e73c2b26c8e6c81`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:31.752665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:31.770320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.771111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.774156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.774156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.774156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.775516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.842698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:31.863392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1097/2201] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:31.868634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.883746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b15e-9e46-48e1-89dc-a8207e04a555 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.888756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.889493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.889847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.890192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.891076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.893437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:31.913447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1105/2202] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/projects => generated 371 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:31.917471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:31.981031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c26b7-2051-4534-a963-4c8365395288 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:31.987234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:31.987918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:31.988570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:31.988775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:31.989417 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:31.991690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.019428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1098/2203] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/projects => generated 401 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:48:32.021038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.108311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1106/2204] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:31 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:48:32.112289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.113556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d39dcb-370c-42a1-8e63-da888acf85ba admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.118208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.118903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.119191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.119469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.120165 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.122896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.127646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f21f-0a1b-4ef9-af41-59d5cbe6dd51 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.128094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.128747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.129031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.129311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.130087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.131940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.142691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1107/2205] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:48:32.146522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.158005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4358e-9e1e-44fe-93af-863c65028503 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.162728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.163385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.163671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.163951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.164671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.166379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.179334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1108/2206] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:48:32.185083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.198277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8d975-41d2-462c-ac05-4597166af447 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.203947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.204845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.205133 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.205408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] PATH_INFO: `/v3/projects/8bd7a87e367b458b97b9b3361c6be53c/users/95655cd0d58342559a43e4e4da0319d1/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.221516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd7a87e367b458b97b9b3361c6be53c, user_id=95655cd0d58342559a43e4e4da0319d1, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.225353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.250650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1099/2207] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/users => generated 307 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:32.250650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.254756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.268171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e941a-91c6-4e49-83d3-2f37c958191b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.273166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.274032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.274367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.274715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] PATH_INFO: `/v3/projects/b0f367d309d74e7eaaa23d732fc7c65a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.275883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f367d309d74e7eaaa23d732fc7c65a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.277864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.346573 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1109/2208] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:32 2022] PUT /identity/v3/projects/8bd7a87e367b458b97b9b3361c6be53c/users/95655cd0d58342559a43e4e4da0319d1/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:32.348948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.410157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f367d309d74e7eaaa23d732fc7c65a', 'request_id': 'req-46f33e7f-3067-4b62-8631-42449e48e225'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:32.420573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:32.427094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a845c3-81c2-4225-b357-83def31f420f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.470227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f367d309d74e7eaaa23d732fc7c65a', 'request_id': 'req-46f33e7f-3067-4b62-8631-42449e48e225'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:32.476560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f367d309d74e7eaaa23d732fc7c65a', 'request_id': 'req-46f33e7f-3067-4b62-8631-42449e48e225'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:32.482804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.483504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.483913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.484113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.484806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.486536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06a82-c285-46bf-b695-51908fd7353f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.490014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] The token cache is being invalidate because project b0f367d309d74e7eaaa23d732fc7c65a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:32.490309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:32.491111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-46f33e7f-3067-4b62-8631-42449e48e225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:32.495281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1100/2209] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:32 2022] DELETE /identity/v3/projects/b0f367d309d74e7eaaa23d732fc7c65a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:32.498453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1110/2210] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:32.504980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.505111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeedb-b5ab-4a76-9766-b693dd6e32a2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.505197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.505197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.505327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeedb-b5ab-4a76-9766-b693dd6e32a2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.505327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeedb-b5ab-4a76-9766-b693dd6e32a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.522160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:32.522418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.532331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1111/2211] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:32.532986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.534229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.537095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-44aeeedb-b5ab-4a76-9766-b693dd6e32a2 None None] MFA Rules not processed for user `95655cd0d58342559a43e4e4da0319d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:32.538329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-44aeeedb-b5ab-4a76-9766-b693dd6e32a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.546350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2c111-179c-4cf5-a7ed-03d2aa91efa5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.551678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.552585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.552866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.553140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] PATH_INFO: `/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7/users/9bb92c7e44a04359b7f06d964d0ccca3/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.568754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=790aaf3c95b8456fa86e4a7ba35c6eb7, user_id=9bb92c7e44a04359b7f06d964d0ccca3, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.574338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.638297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1101/2212] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:32.639992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.640296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.640574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.654317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1112/2213] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:32 2022] PUT /identity/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7/users/9bb92c7e44a04359b7f06d964d0ccca3/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:32.655391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.657221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.667205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:32.669143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.715783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43599a69-faa3-42e9-9f9c-d88dca98b3df admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.720028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.720998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.721332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.721702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.738994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1102/2214] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:32.740181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.750937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930c67f-e021-47b3-a5d2-9324b762bb92 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.756173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.756845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.757116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.757391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.758119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.760018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.771677 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1103/2215] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:32.772570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.782838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84d54-3eae-4427-8021-000a1a287e0c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.787560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.788254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.788487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.788751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.789393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.789517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:32.789765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.790986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.791380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.801611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1113/2216] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:32.802408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.805615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1104/2217] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:32.811788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.815431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9f3-ed9a-4fc4-9900-2d2dd311ad9f service glance] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.819316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:32.822789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5793-261e-43ac-9227-ccb9bbb385f9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:32.829666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.830160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:32.830815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.831122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.831242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.831409 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.831594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.831699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] PATH_INFO: `/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7/users/9bb92c7e44a04359b7f06d964d0ccca3/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.832410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.833910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790aaf3c95b8456fa86e4a7ba35c6eb7, user_id=9bb92c7e44a04359b7f06d964d0ccca3, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:32.834901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.838246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a18b4a-5c18-4edd-8444-5e2402f9d113 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:32.854414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:32.860942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1114/2218] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:32 2022] PUT /identity/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7/users/9bb92c7e44a04359b7f06d964d0ccca3/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:32.875239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:32.875239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:32.875239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:32.914796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] MFA Rules not processed for user `9bb92c7e44a04359b7f06d964d0ccca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:32.916275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:32.950962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1105/2219] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:32.954880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:32.996494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1115/2220] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:33.007079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.009898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:33.026937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc84b-4d70-4f5f-b680-fca465d2aded admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.034934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.039500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.039500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.039500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.039500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.040995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.084618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306202f9-c39a-4d50-828a-48c7825d48b5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.121509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1106/2221] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:32 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:33.125535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.137932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf9b1-3bec-434b-9a86-01cd73def951 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.146511 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.147339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.147646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.147897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.149011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.150683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.152105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.152715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.153012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.153284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.166893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1107/2222] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:33.177925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.185368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbdbd7-9fb6-4f4f-b74e-a37998702de5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.190845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.191648 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.191934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.193009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.193009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.194694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.208624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1108/2223] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:33.212786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.224562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196be68-2c1f-4929-bdf7-eb1ef3d286aa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.229593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.237062 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.237222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.237222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.237222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] PATH_INFO: `/v3/projects/a1c477cba59b42cfb37e996815ee7bf7/users/b4b28d2ab36547ca8c23b3bc292cb384/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.237445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c540-c6e9-4cec-860b-4e814ac95af2 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.244748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1116/2224] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:48:33.247902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c477cba59b42cfb37e996815ee7bf7, user_id=b4b28d2ab36547ca8c23b3bc292cb384, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.250781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.339296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1109/2225] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:33 2022] PUT /identity/v3/projects/a1c477cba59b42cfb37e996815ee7bf7/users/b4b28d2ab36547ca8c23b3bc292cb384/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:33.356609 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.356911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.357198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.370560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:33.415648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] MFA Rules not processed for user `b4b28d2ab36547ca8c23b3bc292cb384`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:33.417328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:33.515584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1117/2226] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:33.536202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.552085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:33.609037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823ae90-075b-406c-a8c4-f42e83e4218c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.613504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.614139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.614423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.614744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.624036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.625480 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.634653 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1110/2227] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:33.660636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.660904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:33.670933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dee89-52e9-4545-bba6-7b09edef9fb9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.674973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.675617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.679318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.679318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.679318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:33.782863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c84499-370c-4491-b6a6-3998b0595a4e service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:33.787652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:33.788530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:33.788837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:33.789122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] PATH_INFO: `/v3/users/b4b28d2ab36547ca8c23b3bc292cb384` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:33.790347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b28d2ab36547ca8c23b3bc292cb384)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.793171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.802912 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:33.806666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba9454-ced1-4077-bfab-c27f635ad6c0 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:33.821159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1118/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:33.905604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:34.059157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b28d2ab36547ca8c23b3bc292cb384'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:34.069232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b28d2ab36547ca8c23b3bc292cb384'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:34.144787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b28d2ab36547ca8c23b3bc292cb384'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:34.145374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b28d2ab36547ca8c23b3bc292cb384'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:34.146275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b28d2ab36547ca8c23b3bc292cb384'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:34.180775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1111/2229] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:33 2022] DELETE /identity/v3/users/b4b28d2ab36547ca8c23b3bc292cb384 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:34.187217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:34.187615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:34.187991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:34.216271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:34.228448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:34.299330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1119/2230] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:34.309771 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:34.311281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:34.380531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eda-5b3f-4af3-ab9e-cd53234f2016 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:34.384294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:34.385022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:34.385308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:34.385580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:34.395933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:34.397667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a9436-37e7-443e-b7ca-11d7b43fb26d service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:34.416564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1112/2231] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:34.467416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:34.819272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:35.021310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.023263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:35.048196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f042ae-9ed2-4e21-8586-2d6fbd71c115 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.056416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.056416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.056416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.056416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] PATH_INFO: `/v3/projects/a1c477cba59b42cfb37e996815ee7bf7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.057052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c477cba59b42cfb37e996815ee7bf7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.059845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:35.192071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c477cba59b42cfb37e996815ee7bf7', 'request_id': 'req-ccc43d37-13b4-4057-8ba8-9c6064027ecc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:35.264139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c477cba59b42cfb37e996815ee7bf7', 'request_id': 'req-ccc43d37-13b4-4057-8ba8-9c6064027ecc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:35.270980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c477cba59b42cfb37e996815ee7bf7', 'request_id': 'req-ccc43d37-13b4-4057-8ba8-9c6064027ecc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:35.282709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] The token cache is being invalidate because project a1c477cba59b42cfb37e996815ee7bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:35.283001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:35.283791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ccc43d37-13b4-4057-8ba8-9c6064027ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:35.285346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1120/2232] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:35 2022] DELETE /identity/v3/projects/a1c477cba59b42cfb37e996815ee7bf7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:35.290449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.290768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.291048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.299474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:35.310314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:35.311457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:35.377871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1113/2233] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:35.385606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0477-167a-4066-b3c3-27832219f56e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.385898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0477-167a-4066-b3c3-27832219f56e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.386180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0477-167a-4066-b3c3-27832219f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.404882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-881e0477-167a-4066-b3c3-27832219f56e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:35.406016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-881e0477-167a-4066-b3c3-27832219f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:35.414768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1121/2234] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:35.421076 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.432035 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419741a-5e32-43b4-95b5-134a7966e2c2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.438908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.439629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.442036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.442036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.442036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:35.442036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.442857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:35.459355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1114/2235] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:35.463630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e0477-167a-4066-b3c3-27832219f56e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.473480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e0477-167a-4066-b3c3-27832219f56e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.478646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.479299 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.479629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.479859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.480825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.482907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:35.576413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1122/2236] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:35 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:35.579916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.590461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675963-aca8-4659-a7ef-04e05e959aed admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.599806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:35.666174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.666893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.668783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.679778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.680663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.689650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:35.868867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1115/2237] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:35 2022] POST /identity/v3/users => generated 295 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:35.870083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.883310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd604b0-ba5b-4cff-acac-86209b99a67c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.904695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.905987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.906513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.909274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.910869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.913316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:35.939638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1123/2238] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:35.940381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:35.952702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c53c-434f-48aa-9a9a-e5ef9e9822bb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:35.959340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:35.964224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:35.964224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:35.964224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:35.964224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:35.964224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19a6c5-cbbc-4213-bfdd-6ab990867afa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:36.005198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1116/2239] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:36.006648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:36.017940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154d710-c84c-4fd1-bb54-8f275ddd2337 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:36.022967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:36.023970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:36.024322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:36.024672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] PATH_INFO: `/v3/projects/a26514486fa444688c657be88a4a16e2/users/7ee3de55b65940b1b573f866d1895272/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:36.044707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26514486fa444688c657be88a4a16e2, user_id=7ee3de55b65940b1b573f866d1895272, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:36.051530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:36.192781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1124/2240] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:36 2022] PUT /identity/v3/projects/a26514486fa444688c657be88a4a16e2/users/7ee3de55b65940b1b573f866d1895272/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:36.207003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:36.207622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:36.207622 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:36.266321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] MFA Rules not processed for user `7ee3de55b65940b1b573f866d1895272`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:36.268136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:36.370045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1117/2241] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:36.382860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:36.384837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:36.507778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77850d56-135e-4ceb-b585-99955f82feb0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:36.521506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:36.522585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:36.522901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:36.523763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:36.538904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:36.541037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab918-c293-4f8d-be0f-a956e31b6607 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:36.555587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1125/2242] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:40.396465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:40.398382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:40.462623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fc3df-220d-4741-8766-a48c0c234cd3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:40.468072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:40.468933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:40.469212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:40.469804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] PATH_INFO: `/v3/users/82163223dae94b158e73c2b26c8e6c81` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:40.470984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82163223dae94b158e73c2b26c8e6c81)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:40.473583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:40.496172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:40.674296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82163223dae94b158e73c2b26c8e6c81'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:40.682203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82163223dae94b158e73c2b26c8e6c81'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:40.744428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82163223dae94b158e73c2b26c8e6c81'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:40.745071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82163223dae94b158e73c2b26c8e6c81'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:40.745736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82163223dae94b158e73c2b26c8e6c81'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:40.764256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1118/2243] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:40 2022] DELETE /identity/v3/users/82163223dae94b158e73c2b26c8e6c81 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:40.768851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:40.769352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:40.769447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:40.788356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:40.789582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:40.852466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1126/2244] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:40.863947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:40.865733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:40.931611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f95-731d-497b-b0b1-8e558c07003d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:40.938780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:40.938780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:40.938780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:40.938780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:40.947516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:40.949764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:40.960931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1119/2245] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:41.478139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:41.479672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:41.494783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ce1f-946b-45d9-91d7-4b61cf3fd350 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:41.500305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:41.502322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:41.502322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:41.502322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] PATH_INFO: `/v3/projects/6b1f4202d97044efb0af8cbd33213911` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:41.503077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1f4202d97044efb0af8cbd33213911)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:41.505022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:41.528471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:41.659166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1f4202d97044efb0af8cbd33213911', 'request_id': 'req-5b4a2113-2327-4c27-94a7-341c99cf528c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:41.727214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1f4202d97044efb0af8cbd33213911', 'request_id': 'req-5b4a2113-2327-4c27-94a7-341c99cf528c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:41.735335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1f4202d97044efb0af8cbd33213911', 'request_id': 'req-5b4a2113-2327-4c27-94a7-341c99cf528c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:41.753231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] The token cache is being invalidate because project 6b1f4202d97044efb0af8cbd33213911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:41.753568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:41.754433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:41.757093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1127/2246] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:41 2022] DELETE /identity/v3/projects/6b1f4202d97044efb0af8cbd33213911 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:41.760153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:41.770095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e3b5-f00c-4db3-8abd-bf0a93d7bd93 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:41.832234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:41.833118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:41.833400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:41.833676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] PATH_INFO: `/v3/users/9bb92c7e44a04359b7f06d964d0ccca3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:41.835014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb92c7e44a04359b7f06d964d0ccca3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:41.836954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:42.052053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:42.056671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb92c7e44a04359b7f06d964d0ccca3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.064533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb92c7e44a04359b7f06d964d0ccca3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.138453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb92c7e44a04359b7f06d964d0ccca3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.139151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb92c7e44a04359b7f06d964d0ccca3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.139786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4e0fc977-a0a5-46d9-b23b-7ea060e6e537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb92c7e44a04359b7f06d964d0ccca3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.167532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1120/2247] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:41 2022] DELETE /identity/v3/users/9bb92c7e44a04359b7f06d964d0ccca3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:42.671788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:42.674610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:42.688272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2113-2327-4c27-94a7-341c99cf528c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:42.754580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:42.755462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:42.755745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:42.756026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] PATH_INFO: `/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:42.757226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790aaf3c95b8456fa86e4a7ba35c6eb7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:42.760546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:42.877512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790aaf3c95b8456fa86e4a7ba35c6eb7', 'request_id': 'req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.950337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790aaf3c95b8456fa86e4a7ba35c6eb7', 'request_id': 'req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.957800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790aaf3c95b8456fa86e4a7ba35c6eb7', 'request_id': 'req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.969282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] The token cache is being invalidate because project 790aaf3c95b8456fa86e4a7ba35c6eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:42.969837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.970499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd24a7d1-06dd-4470-9d0f-7c573d8121f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:42.972242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1128/2248] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:42 2022] DELETE /identity/v3/projects/790aaf3c95b8456fa86e4a7ba35c6eb7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:42.977322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:42.977923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:42.977923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.007436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:43.008645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:43.077082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1121/2249] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:43.085054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.085341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.085620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.106080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:43.107255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:43.115320 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1129/2250] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:43.119695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.130022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72013b-4b92-4eb7-b2f3-2f61009e5142 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.134764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.135398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.135678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.136569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.140869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:43.140869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.140869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.152224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1122/2251] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:43.156817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.167127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6a326-2314-42a9-a7e5-9b2c012a8312 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.171866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.172516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.172797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.173073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.173769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.175703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.282691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1130/2252] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:43 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:43.286949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.297841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3c4e-31c4-4578-9c76-68b3adaba4b0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.374849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.375540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.375823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.376097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.376795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.379704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.470498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1123/2253] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:43 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:43.471440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.484589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67b9c-c0a6-47fb-87de-fdb5bba14cb5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.489408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.490067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.490350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.490697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.491458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.493017 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.495230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:43.504617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1131/2254] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:48:43.509104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.519954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d636b-553e-48cb-9793-5002f33609ae admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.526647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.526647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.526647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.526647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.527294 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.529101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2905ed8-7319-4354-bc69-da14cd8db882 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.539835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1124/2255] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:43.544659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.554872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df5ce-833e-41f8-b244-4da1bf4abd42 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.560203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.561360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.561360 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.561636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] PATH_INFO: `/v3/projects/8bf568047aea4dc0a7c971faaed85433/users/78ae7e323ea2496596022e40a060c806/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.578023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf568047aea4dc0a7c971faaed85433, user_id=78ae7e323ea2496596022e40a060c806, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.581233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:43.657326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1132/2256] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:43 2022] PUT /identity/v3/projects/8bf568047aea4dc0a7c971faaed85433/users/78ae7e323ea2496596022e40a060c806/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:43.671649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.671969 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.672210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.720588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] MFA Rules not processed for user `78ae7e323ea2496596022e40a060c806`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:43.722206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:43.775124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:43.858342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1125/2257] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:43.863179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:43.938888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34596-eb72-46c6-9200-6523f189c17c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:43.943461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:43.944116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:43.944396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:43.944678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:43.945352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:43.947293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.176667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1133/2258] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:44.178911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.190516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4f11-0c3d-4f7d-bce2-8b0b77ed8d9f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.250730 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.251460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.251741 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.252019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.252840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.255058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.368631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1126/2259] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:44 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:44.373200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.383499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282ff0-dfd6-4e25-854e-c5b3cbd4dcde admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.388710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.389372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.389648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.389923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.390680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.392405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.403570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1134/2260] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:44.407698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.419610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148d28c-171d-439e-b0d7-f815ada363b2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.424317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.425229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.425519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.425775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] PATH_INFO: `/v3/projects/cec0e4d7147d4d059861a47176cef54b/users/ba4b985e6bda4d60b83f9bfad47ea508/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.442320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec0e4d7147d4d059861a47176cef54b, user_id=ba4b985e6bda4d60b83f9bfad47ea508, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.444951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.574484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1127/2261] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:44 2022] PUT /identity/v3/projects/cec0e4d7147d4d059861a47176cef54b/users/ba4b985e6bda4d60b83f9bfad47ea508/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:44.578615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.588879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93ea4-2c0e-4039-9c28-81f574960991 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.649987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.650282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.650587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.650954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.651733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.653477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a8da0-112d-4dd2-8351-a7a081d518ff admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.674962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1135/2262] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:48:44.675067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.681497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039f273-b8b6-42ce-93a8-b6c0500bf356 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.687475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.688386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.688665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.688938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] PATH_INFO: `/v3/projects/cec0e4d7147d4d059861a47176cef54b/users/ba4b985e6bda4d60b83f9bfad47ea508/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.690914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec0e4d7147d4d059861a47176cef54b, user_id=ba4b985e6bda4d60b83f9bfad47ea508, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.693678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.706477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1128/2263] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:44 2022] PUT /identity/v3/projects/cec0e4d7147d4d059861a47176cef54b/users/ba4b985e6bda4d60b83f9bfad47ea508/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:44.721052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.721344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.721627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.751121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] MFA Rules not processed for user `ba4b985e6bda4d60b83f9bfad47ea508`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:44.752354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:44.785315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:44.833031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1136/2264] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:44.847978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:44.849711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:44.918516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8354-0758-4c27-b308-6c9f9a7e4382 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:44.922759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:44.923642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:44.923922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:44.924199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:44.936102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:44.937615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:44.949126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1129/2265] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:45.107486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:45.118851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe1edc-865e-4374-9adc-6241c200d737 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:45.123583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:45.124218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:45.124594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:45.124933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:45.184871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:45.186334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca28b3-7245-4068-bed7-25b3962a16e0 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:45.196187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1137/2266] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:45.679928 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:45.690716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:45.751911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949ba6-9abc-4b76-b254-d5e2c394aff8 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:45.756513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:45.757471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:45.757763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:45.758042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] PATH_INFO: `/v3/users/78ae7e323ea2496596022e40a060c806` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:45.760752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ae7e323ea2496596022e40a060c806)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:45.761600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:45.968530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ae7e323ea2496596022e40a060c806'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:45.976221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ae7e323ea2496596022e40a060c806'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.069726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ae7e323ea2496596022e40a060c806'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.070351 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ae7e323ea2496596022e40a060c806'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.070987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ae7e323ea2496596022e40a060c806'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.091503 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1130/2267] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:45 2022] DELETE /identity/v3/users/78ae7e323ea2496596022e40a060c806 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:46.098043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:46.098043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:46.098043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:46.115774 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:46.116964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:46.180445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1138/2268] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:46.191164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:46.192612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:46.254929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0e29-2b11-43ed-89ff-de42f9db343a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:46.258610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:46.259264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:46.259591 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:46.259897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:46.269707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:46.271263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:46.280912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1131/2269] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:46.757100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:46.758677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:46.771776 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90084138-ec4d-43b0-8acd-bdfbeb383d22 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:46.776432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:46.777815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:46.778102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:46.778378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] PATH_INFO: `/v3/projects/8bf568047aea4dc0a7c971faaed85433` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:46.779599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf568047aea4dc0a7c971faaed85433)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:46.781561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:46.924024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf568047aea4dc0a7c971faaed85433', 'request_id': 'req-079c4b51-f051-446c-be4a-421d860b56d3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.983996 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf568047aea4dc0a7c971faaed85433', 'request_id': 'req-079c4b51-f051-446c-be4a-421d860b56d3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.991503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf568047aea4dc0a7c971faaed85433', 'request_id': 'req-079c4b51-f051-446c-be4a-421d860b56d3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:46.993879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:47.013675 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] The token cache is being invalidate because project 8bf568047aea4dc0a7c971faaed85433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:47.013974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.014720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.016745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1139/2270] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:46 2022] DELETE /identity/v3/projects/8bf568047aea4dc0a7c971faaed85433 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:47.020574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:47.043658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8759d92-5fd4-4b2a-8e9b-b1b8603fb097 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:47.077151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:47.111280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:47.112182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:47.112555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:47.112901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] PATH_INFO: `/v3/users/ba4b985e6bda4d60b83f9bfad47ea508` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:47.114532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4b985e6bda4d60b83f9bfad47ea508)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:47.116447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:47.319833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4b985e6bda4d60b83f9bfad47ea508'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.333318 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4b985e6bda4d60b83f9bfad47ea508'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.402214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4b985e6bda4d60b83f9bfad47ea508'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.402923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4b985e6bda4d60b83f9bfad47ea508'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.403614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a146d05f-fc02-4a71-80a7-8b3d72ac1ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4b985e6bda4d60b83f9bfad47ea508'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:47.421517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1132/2271] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:47 2022] DELETE /identity/v3/users/ba4b985e6bda4d60b83f9bfad47ea508 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:47.941119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:47.942738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:47.964002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4b51-f051-446c-be4a-421d860b56d3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.030174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.031112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.031398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.033146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] PATH_INFO: `/v3/projects/cec0e4d7147d4d059861a47176cef54b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.033146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec0e4d7147d4d059861a47176cef54b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.035135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.217710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec0e4d7147d4d059861a47176cef54b', 'request_id': 'req-08437961-98c4-4a85-bc95-df83db8f4816'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:48.286809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec0e4d7147d4d059861a47176cef54b', 'request_id': 'req-08437961-98c4-4a85-bc95-df83db8f4816'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:48.293511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec0e4d7147d4d059861a47176cef54b', 'request_id': 'req-08437961-98c4-4a85-bc95-df83db8f4816'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:48.304029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] The token cache is being invalidate because project cec0e4d7147d4d059861a47176cef54b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:48.304316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:48.304952 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08437961-98c4-4a85-bc95-df83db8f4816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:48.307434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1140/2272] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:47 2022] DELETE /identity/v3/projects/cec0e4d7147d4d059861a47176cef54b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:48.318093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.318639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.318849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.359763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:48.361000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:48.430361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1133/2273] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:48.432482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.432804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.433118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.451986 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:48.453191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:48.461146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1141/2274] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:48.466569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.476889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f972178-2f94-46d8-af4c-e0e6395ca199 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.482171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.483364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.483364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.483560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.484149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:48:48.484458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.486055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.498656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1134/2275] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:48:48.504703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.515303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcb677-cc6a-48b9-b645-100db564254c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.520398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.521048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.521329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.521603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.522297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.524311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.628493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1142/2276] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:48 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:48.630104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.640933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcef92-3028-4a6b-828a-6c7ba7e07531 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.674497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:48.697379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.697961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.698817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.699071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.699768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.701717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.830290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1135/2277] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:48 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:48.834608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.845946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce9-d030-429e-867a-1a8d4efa977a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.850533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.851278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.851564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.851844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.852564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.854165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.866176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1143/2278] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:48.871562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.881963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c186d-8e89-4be2-ae47-133f2432c2c9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.888028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.888699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.888981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.889254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.889977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.891767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cce218-d082-4197-9807-dd96e86a7640 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:48.906105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1136/2279] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:48.911243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:48.917719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0974-b286-441d-9ab2-03b67ebdc714 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:48.922781 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:48.923667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:48.923951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:48.924253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] PATH_INFO: `/v3/projects/9224bcedeba54c8fbe1cafb502386b8b/users/d2e3c6ccaa504b2dab9fd2160150eb23/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:48.942431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9224bcedeba54c8fbe1cafb502386b8b, user_id=d2e3c6ccaa504b2dab9fd2160150eb23, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:48.945152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.041610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1144/2280] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:48 2022] PUT /identity/v3/projects/9224bcedeba54c8fbe1cafb502386b8b/users/d2e3c6ccaa504b2dab9fd2160150eb23/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:49.052109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.052707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.053064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.082449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] MFA Rules not processed for user `d2e3c6ccaa504b2dab9fd2160150eb23`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:49.083734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:49.150084 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1137/2281] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:49.154785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.174144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:49.216880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d406-bc3f-4d01-8458-42d26c2d57a4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.221682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.223391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.223391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.223391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.223633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.225455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.294858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1145/2282] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:48:49 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:49.298945 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.310499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1405b0-f7d9-4794-934c-a7014ba3e246 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.372897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.373656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.373935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.374207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.374934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.376842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.458237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1138/2283] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:48:49 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:48:49.462164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.473156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1958-51bb-478d-b906-ec0b50c0a4a1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.477837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.478488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.478808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.479089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.480422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.482131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.493620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1146/2284] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:49.497866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.507894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67878-6151-43ef-a029-70365dfa52b2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.513345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.514224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.514500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.514801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] PATH_INFO: `/v3/projects/021e86470d7d4de6ad4ce667264e5635/users/8c94171119074c9dafbe1079d2ee3b02/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.530685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=021e86470d7d4de6ad4ce667264e5635, user_id=8c94171119074c9dafbe1079d2ee3b02, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.533320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.601855 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1139/2285] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:49 2022] PUT /identity/v3/projects/021e86470d7d4de6ad4ce667264e5635/users/8c94171119074c9dafbe1079d2ee3b02/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:49.605922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.616376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92baf-287f-47c9-8f06-1231eee9a56f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.683863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.684601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.684810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.685093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.685820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.687445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0836fad-510c-4cb3-a486-7b448da5ace6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.700754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1147/2286] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:48:49.701633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:49.711878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778502d-bcb0-48c5-abd8-7b087e1010cb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:49.717510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:49.718421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.718820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.719095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] PATH_INFO: `/v3/projects/021e86470d7d4de6ad4ce667264e5635/users/8c94171119074c9dafbe1079d2ee3b02/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.721673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=021e86470d7d4de6ad4ce667264e5635, user_id=8c94171119074c9dafbe1079d2ee3b02, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:49.724354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:49.732311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:49.737930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1140/2287] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:48:49 2022] PUT /identity/v3/projects/021e86470d7d4de6ad4ce667264e5635/users/8c94171119074c9dafbe1079d2ee3b02/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:48:49.750960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:49.751312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:49.751578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:49.780988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] MFA Rules not processed for user `8c94171119074c9dafbe1079d2ee3b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:49.782376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:49.855948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1148/2288] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:50.447025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:50.448806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:50.509281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826df0-39c3-47b5-af1b-ed9731154da3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:50.513843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:50.514757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:50.515110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:50.515457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:50.579862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:50.581313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b85b2-94db-4a30-a49d-6113318e44e9 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:50.595566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1141/2289] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:55.771253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:55.784791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:55.835742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa78b60-6419-4a05-9207-f6fe61accf97 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:55.840850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:55.841801 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:55.842185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:55.842769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] PATH_INFO: `/v3/users/d2e3c6ccaa504b2dab9fd2160150eb23` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:55.845088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e3c6ccaa504b2dab9fd2160150eb23)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:55.848064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:56.090671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e3c6ccaa504b2dab9fd2160150eb23'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:56.098425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e3c6ccaa504b2dab9fd2160150eb23'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:56.099185 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e3c6ccaa504b2dab9fd2160150eb23'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:56.099886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e3c6ccaa504b2dab9fd2160150eb23'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:56.167187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e3c6ccaa504b2dab9fd2160150eb23'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:56.189325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1149/2290] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:55 2022] DELETE /identity/v3/users/d2e3c6ccaa504b2dab9fd2160150eb23 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:56.194388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:56.195100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:56.197122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:56.216338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:56.217667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:56.285414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1142/2291] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:48:56.296454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:56.297908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:56.378332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaaad1-599a-4d05-b5d2-2c61cd08deed admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:56.382328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:56.383027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:56.383312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:56.383590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:56.395457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:56.397421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:56.409407 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1150/2292] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:48:56.840672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:56.842628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:56.872014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806aa4-4cae-4b93-8e8f-91dea4e48c15 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:56.875491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:56.876430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:56.876711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:56.876983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] PATH_INFO: `/v3/projects/9224bcedeba54c8fbe1cafb502386b8b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:56.878256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9224bcedeba54c8fbe1cafb502386b8b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:56.880228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:57.018491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:57.020450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9224bcedeba54c8fbe1cafb502386b8b', 'request_id': 'req-80d9b848-efe7-4136-a7a2-732333ec43a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.092481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9224bcedeba54c8fbe1cafb502386b8b', 'request_id': 'req-80d9b848-efe7-4136-a7a2-732333ec43a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.099407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9224bcedeba54c8fbe1cafb502386b8b', 'request_id': 'req-80d9b848-efe7-4136-a7a2-732333ec43a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.111803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] The token cache is being invalidate because project 9224bcedeba54c8fbe1cafb502386b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:57.112120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.112829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.114374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1143/2293] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:56 2022] DELETE /identity/v3/projects/9224bcedeba54c8fbe1cafb502386b8b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:57.118644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:57.130835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9719cd-e832-446a-8801-eb447b6aa996 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:57.167514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:57.200272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:57.201162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:57.201446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:57.201725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] PATH_INFO: `/v3/users/8c94171119074c9dafbe1079d2ee3b02` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:57.203033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c94171119074c9dafbe1079d2ee3b02)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:57.204894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:57.423750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c94171119074c9dafbe1079d2ee3b02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.430919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c94171119074c9dafbe1079d2ee3b02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.431574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c94171119074c9dafbe1079d2ee3b02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.432197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c94171119074c9dafbe1079d2ee3b02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.514208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c94171119074c9dafbe1079d2ee3b02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:57.576602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1151/2294] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:57 2022] DELETE /identity/v3/users/8c94171119074c9dafbe1079d2ee3b02 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:58.264253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:58.266937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:58.282144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b848-efe7-4136-a7a2-732333ec43a2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:58.348381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:58.349387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:58.349739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:58.350093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] PATH_INFO: `/v3/projects/021e86470d7d4de6ad4ce667264e5635` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:58.351533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021e86470d7d4de6ad4ce667264e5635)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:58.353569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:58.472917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021e86470d7d4de6ad4ce667264e5635', 'request_id': 'req-f19c0750-e118-4b53-b6e7-3ef71ca4c559'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:58.612882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021e86470d7d4de6ad4ce667264e5635', 'request_id': 'req-f19c0750-e118-4b53-b6e7-3ef71ca4c559'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:58.621751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021e86470d7d4de6ad4ce667264e5635', 'request_id': 'req-f19c0750-e118-4b53-b6e7-3ef71ca4c559'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:58.635524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] The token cache is being invalidate because project 021e86470d7d4de6ad4ce667264e5635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:48:58.636487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:58.637773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f19c0750-e118-4b53-b6e7-3ef71ca4c559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:58.640736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1144/2295] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:48:58 2022] DELETE /identity/v3/projects/021e86470d7d4de6ad4ce667264e5635 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:59.028245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:59.030637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:59.047566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1352abf-ddca-41a3-b09a-78d317830ed8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:59.140746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:59.141849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:59.142881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:59.143180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] PATH_INFO: `/v3/users/7ee3de55b65940b1b573f866d1895272` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:59.144602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee3de55b65940b1b573f866d1895272)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:59.146757 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:59.310999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee3de55b65940b1b573f866d1895272'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:59.318902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee3de55b65940b1b573f866d1895272'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:59.319703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee3de55b65940b1b573f866d1895272'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:59.320332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee3de55b65940b1b573f866d1895272'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:59.322228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:48:59.394375 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee3de55b65940b1b573f866d1895272'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:48:59.413308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1152/2296] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:48:59 2022] DELETE /identity/v3/users/7ee3de55b65940b1b573f866d1895272 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:48:59.418319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:59.418880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:59.418880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:59.437828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:48:59.442617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:59.501724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1145/2297] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:48:59.511422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:48:59.513469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:48:59.574122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d631-8da7-4d51-b452-abffca634c5b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:48:59.578420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:48:59.579104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:48:59.579408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:48:59.579683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:48:59.588870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:48:59.590521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a337-82bc-40dc-82da-f711174353ef service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:48:59.602281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1153/2298] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:00.346179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:00.348300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:00.364334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72843d35-3fa1-433c-977f-ad4a5faa2c16 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:00.370429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:00.371517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:00.372667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:00.373244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] PATH_INFO: `/v3/projects/a26514486fa444688c657be88a4a16e2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:00.375116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26514486fa444688c657be88a4a16e2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:00.377277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:00.379270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:00.492376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26514486fa444688c657be88a4a16e2', 'request_id': 'req-03b2d868-3da5-4c95-ac0a-7ac97ff95026'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:00.599934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26514486fa444688c657be88a4a16e2', 'request_id': 'req-03b2d868-3da5-4c95-ac0a-7ac97ff95026'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:00.611647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26514486fa444688c657be88a4a16e2', 'request_id': 'req-03b2d868-3da5-4c95-ac0a-7ac97ff95026'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:00.624424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] The token cache is being invalidate because project a26514486fa444688c657be88a4a16e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:49:00.624717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:00.625449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-03b2d868-3da5-4c95-ac0a-7ac97ff95026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:00.627181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1146/2299] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:49:00 2022] DELETE /identity/v3/projects/a26514486fa444688c657be88a4a16e2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:00.632281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:00.632779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:00.632886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:00.652839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:00.654853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:00.718347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1154/2300] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:00.726366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:00.726684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:00.726963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:00.745317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:00.746509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:00.754402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1147/2301] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:00.758668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:00.771731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37cc7c-7984-4c4d-ba41-d16f2fa69bf1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:00.774778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:00.775444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:00.775730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:00.776008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:00.776842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:49:00.777033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:00.779403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:00.791297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1155/2302] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:49:00.795837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:00.805278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f376b0-f3fe-4b6d-ad16-c327925ead81 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:00.810809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:00.811274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:00.811551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:00.811823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:00.812534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:00.814430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:00.932947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1148/2303] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:49:00 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:00.937981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:00.948348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32413d36-a006-4089-b384-b34bc1156e00 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:01.005162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:01.005830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.006111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.006385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.007101 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:01.009000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:01.018314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:01.127852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1156/2304] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:49:00 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:49:01.132206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:01.143303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0e3a-6266-41f6-a499-0c32ee616b6f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:01.147879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:01.148669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.149202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.149887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.151098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:01.154214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:01.166929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1149/2305] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:01.171198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:01.180171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3540d94-bcf8-48bd-bde1-4a735586b4f4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:01.185832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:01.186628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.186775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.187053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.187810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:01.189388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f328e-f72a-4a97-b0fc-b080fb83b896 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:01.201669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1157/2306] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:01.206110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:01.216467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab878395-4a95-48c4-96fc-1f9579bd0072 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:01.221222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:01.222086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.222494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.223038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] PATH_INFO: `/v3/projects/620ea9c4eaa74dc5858cf93d9fcad9f1/users/e2e3fd195fd14b009ba56950dc542cb1/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.240176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620ea9c4eaa74dc5858cf93d9fcad9f1, user_id=e2e3fd195fd14b009ba56950dc542cb1, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:01.242872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:01.303133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1150/2307] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:01 2022] PUT /identity/v3/projects/620ea9c4eaa74dc5858cf93d9fcad9f1/users/e2e3fd195fd14b009ba56950dc542cb1/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:01.308174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba6e4-dc49-4c8a-8c9e-302690e69f63 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.311645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba6e4-dc49-4c8a-8c9e-302690e69f63 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.311645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba6e4-dc49-4c8a-8c9e-302690e69f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.328719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c08ba6e4-dc49-4c8a-8c9e-302690e69f63 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:01.330001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c08ba6e4-dc49-4c8a-8c9e-302690e69f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:01.394841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1158/2308] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:01.404690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:01.406412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:01.471831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecc278-6d33-44b0-a6e8-e6c2d2735840 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:01.475770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:01.476429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:01.476714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:01.477333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:01.488268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:01.489710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:01.500059 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1151/2309] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:49:07.384787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:07.385228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:07.385406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:07.433795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] MFA Rules not processed for user `e2e3fd195fd14b009ba56950dc542cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:07.435717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:07.505566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1159/2310] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:07.510195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:07.521030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15937-f3b6-40c4-82c3-b366743b0c4f service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:07.526093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:07.526809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:07.527094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:07.527377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:07.528762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:07.530796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:07.617757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1152/2311] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:49:07 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:07.622166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:07.640613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c14c4-f4ea-4ad5-be2d-5de99a27ca6f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:07.728833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:07.729857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:07.729995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:07.730215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:07.731906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:07.734594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:07.841054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1160/2312] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:49:07 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:07.846125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:07.856904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55549cc9-7634-4422-a48e-0ca0a4c8f5b9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:07.862722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:07.863497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:07.863895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:07.864237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:07.865157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:07.867091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:07.868725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:07.879666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1153/2313] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:49:07.883981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:07.890275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:07.896288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2686-fa4b-4cf4-afa5-29c110b12c5b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:07.901300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:07.902260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:07.902670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:07.903046 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] PATH_INFO: `/v3/projects/7e7cda367f7b44fc80a663a162608909/users/22cde1a9790e465faea5c592c26df4a0/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:07.919126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7cda367f7b44fc80a663a162608909, user_id=22cde1a9790e465faea5c592c26df4a0, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:07.921949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:08.018965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1161/2314] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:07 2022] PUT /identity/v3/projects/7e7cda367f7b44fc80a663a162608909/users/22cde1a9790e465faea5c592c26df4a0/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:08.024085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:08.033885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5324d3-d499-4b6e-947a-0ca55b3ede41 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:08.096027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:08.096703 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:08.096989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:08.097267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:08.098005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:08.099642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0cbb1-c498-4ea2-a99a-cd4f15a2aae1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:08.113736 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1154/2315] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:08.115010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:08.127192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e7247-b922-4de3-bd70-139993051280 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:08.132217 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:08.133122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:08.133408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:08.133697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] PATH_INFO: `/v3/projects/7e7cda367f7b44fc80a663a162608909/users/22cde1a9790e465faea5c592c26df4a0/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:08.136031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7cda367f7b44fc80a663a162608909, user_id=22cde1a9790e465faea5c592c26df4a0, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:08.139095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:08.149920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1162/2316] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:08 2022] PUT /identity/v3/projects/7e7cda367f7b44fc80a663a162608909/users/22cde1a9790e465faea5c592c26df4a0/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:09.336788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:09.337987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:09.337987 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:09.357867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:09.359194 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:09.435325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1155/2317] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:09.490440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:09.490440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:09.644535 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7af4-285b-4232-9fe8-1c7707b6bafd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:09.662090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:09.663156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:09.663737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:09.664658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:09.687252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:09.696831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460abbb-92b1-4ea8-b884-a070866a4364 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:09.715575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1163/2318] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:49:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:49:14.144459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:14.147267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:14.302211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d0c88-e76a-4e7a-9fd9-27f33266ca96 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:14.313379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:14.314350 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:14.319146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:14.322078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] PATH_INFO: `/v3/users/95655cd0d58342559a43e4e4da0319d1` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:14.322909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95655cd0d58342559a43e4e4da0319d1)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:14.327030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:14.575620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74242f8d-b05a-44f7-9e9b-cf0e72526c2d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:14.576224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74242f8d-b05a-44f7-9e9b-cf0e72526c2d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:14.576686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74242f8d-b05a-44f7-9e9b-cf0e72526c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:14.613412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-74242f8d-b05a-44f7-9e9b-cf0e72526c2d None None] MFA Rules not processed for user `906e4bc4e9cc45ecb01cfca7918e461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:14.615199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-74242f8d-b05a-44f7-9e9b-cf0e72526c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:14.674799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:14.690366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95655cd0d58342559a43e4e4da0319d1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:14.712800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95655cd0d58342559a43e4e4da0319d1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:14.776632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1164/2319] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:49:14.817734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95655cd0d58342559a43e4e4da0319d1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:14.818379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95655cd0d58342559a43e4e4da0319d1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:14.819050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95655cd0d58342559a43e4e4da0319d1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:14.878492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1156/2320] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:49:14 2022] DELETE /identity/v3/users/95655cd0d58342559a43e4e4da0319d1 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:14.882969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:14.883389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:14.884524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:14.944127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:14.945717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:15.043085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:15.056693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1165/2321] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:15.073214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:15.077943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:15.159498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e95db8-b3fa-4766-b0db-b86a3c19d809 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:15.172900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:15.173932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:15.174498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:15.175030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:15.197723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:15.201365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a9adf-2cec-4047-9268-9831372226b1 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:15.213050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1157/2322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:16.329365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:16.330973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:16.347771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274762e-3358-4e1b-bfa7-7643007a311e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:16.353525 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:16.354570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:16.354926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:16.355210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] PATH_INFO: `/v3/projects/8bd7a87e367b458b97b9b3361c6be53c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:16.357012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd7a87e367b458b97b9b3361c6be53c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:16.359458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:16.486222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd7a87e367b458b97b9b3361c6be53c', 'request_id': 'req-2a6603c8-3fee-4996-8930-b7dae3ce2704'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:16.572842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd7a87e367b458b97b9b3361c6be53c', 'request_id': 'req-2a6603c8-3fee-4996-8930-b7dae3ce2704'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:16.588552 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd7a87e367b458b97b9b3361c6be53c', 'request_id': 'req-2a6603c8-3fee-4996-8930-b7dae3ce2704'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:16.607547 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] The token cache is being invalidate because project 8bd7a87e367b458b97b9b3361c6be53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:49:16.607846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:16.608574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a6603c8-3fee-4996-8930-b7dae3ce2704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:49:16.611406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1166/2323] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:49:16 2022] DELETE /identity/v3/projects/8bd7a87e367b458b97b9b3361c6be53c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:16.620210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:16.620210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:16.620210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:16.648479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:16.651638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:16.820316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1158/2324] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:16.821380 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:16.821830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:16.822044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:16.891273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:16.893248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:16.912997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:16.920301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1167/2325] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:16.925574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323cfc9-75a3-4dd8-96be-762e1ec56743 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:16.935335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:16.940391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:16.941021 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:16.941594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:16.943498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:49:16.944172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:16.946203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:16.979146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:16.987737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1159/2326] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:49:16.997498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de77d7-f844-4fa0-8002-012e2e9dccd6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.009714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.010790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.011350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.013176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.019507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.021834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248afe-5d01-4497-91c5-b61fa31ed8c4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.171457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:17.201256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1168/2327] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:49:16 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:17.213960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b68f-5907-41a2-93a2-19470b9f4778 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.296706 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:17.327468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.328173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.328458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.328739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.329448 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.331607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.374080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.375172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.375465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.419296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] MFA Rules not processed for user `22cde1a9790e465faea5c592c26df4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:17.420747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:17.442743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1160/2328] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:49:17 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:17.446851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:17.471099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048d5c7-c0c9-4069-9c3f-350a194fc1aa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.511127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.513643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.514069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.514356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.515289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.517281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.538593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1161/2329] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:17.542354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:17.617030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9e52c-813b-416c-8162-29bfac97b4b3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.619712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1169/2330] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:49:17.636183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:17.641040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:17.654922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.659138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.659544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.664499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.664499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.667142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.695220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1162/2331] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:17.699176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:17.712995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319702fb-d8d9-4e27-b6da-0894b929f290 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.725644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.727679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.728409 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.729227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:17.729764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] PATH_INFO: `/v3/projects/2eff05785bd84522ad637d8bed14563d/users/4ae0271efe4b4e9592b29d9f124519b7/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.756361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7171d3-8ef8-4f13-929f-76acedef8b16 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:17.764467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:17.765858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.766297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.766949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.769389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:17.770042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eff05785bd84522ad637d8bed14563d, user_id=4ae0271efe4b4e9592b29d9f124519b7, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.776598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb872a1-78e3-4dad-a23d-8a217843483a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.901312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1163/2332] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:17 2022] PUT /identity/v3/projects/2eff05785bd84522ad637d8bed14563d/users/4ae0271efe4b4e9592b29d9f124519b7/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:17.916317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:17.917382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b875-b5b1-4c19-8392-16fd95c33013 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:17.917863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b875-b5b1-4c19-8392-16fd95c33013 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:17.918909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b875-b5b1-4c19-8392-16fd95c33013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:17.919151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:17.941104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1170/2333] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:17.951819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-db35b875-b5b1-4c19-8392-16fd95c33013 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:17.953566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-db35b875-b5b1-4c19-8392-16fd95c33013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:18.094859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1164/2334] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:18.101979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:18.108165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:18.239159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da469d0-abcd-4120-9f17-dc859d09c3d1 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:18.322816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:18.323601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:18.323890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:18.324164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:18.348344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:18.349992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8327e2-070e-4a7b-b127-ac124d7a7171 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:18.379949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1171/2335] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:21.633509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:21.634031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:21.634198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:21.663696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] MFA Rules not processed for user `e2e3fd195fd14b009ba56950dc542cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:21.665152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:21.769582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1165/2336] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:28.361940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:28.978699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:28.978868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:28.979164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.048669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] MFA Rules not processed for user `4ae0271efe4b4e9592b29d9f124519b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:29.050242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:29.171968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1172/2337] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:29.176184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.195369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66de9a-0b86-4d3a-aaea-2324cfdc8317 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.200939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.202451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.203478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.204493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.205221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.207292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.334860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1166/2338] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:49:29 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:49:29.339336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.352969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7419338-a46a-408a-9e2f-ad64d3cd74b1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.422520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.423236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.423488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.423761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.424630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.426614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.523354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1173/2339] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:49:29 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:49:29.528234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.539543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81b59-49cf-4a64-bcf6-73f72c24ad99 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.545134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.545885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.546167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.546441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.547272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.549119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.566050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1167/2340] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:29.567351 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.577721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c52d-bca1-453d-a90d-cc71fdda988d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.582501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.583533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.583818 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.584093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] PATH_INFO: `/v3/projects/2b05809a5167425aaee1fe39efa5fd4d/users/9493f392c11542bcb36d37d3c8c18cbb/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.606279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b05809a5167425aaee1fe39efa5fd4d, user_id=9493f392c11542bcb36d37d3c8c18cbb, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.609106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.723120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1174/2341] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:29 2022] PUT /identity/v3/projects/2b05809a5167425aaee1fe39efa5fd4d/users/9493f392c11542bcb36d37d3c8c18cbb/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:29.727198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.733155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:29.740600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373421-31a0-4c9f-baa2-1f73d9ad862d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.809909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.810617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.810901 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.811175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.811905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.813628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b47f4c-d5de-48d5-82db-a0792c2aaf4c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.825121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1168/2342] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:49:29.829348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:29.840702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d42987-bd84-4129-8aa5-d3b87043cacf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:29.846914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:29.847800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:29.848379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:29.848664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] PATH_INFO: `/v3/projects/2b05809a5167425aaee1fe39efa5fd4d/users/9493f392c11542bcb36d37d3c8c18cbb/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:29.850816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b05809a5167425aaee1fe39efa5fd4d, user_id=9493f392c11542bcb36d37d3c8c18cbb, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:29.853765 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:29.871361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1175/2343] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:49:29 2022] PUT /identity/v3/projects/2b05809a5167425aaee1fe39efa5fd4d/users/9493f392c11542bcb36d37d3c8c18cbb/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:49:36.586960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956fdd4-6799-4d61-9651-f3c3f9af4d4d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:36.587829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956fdd4-6799-4d61-9651-f3c3f9af4d4d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:36.588172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956fdd4-6799-4d61-9651-f3c3f9af4d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:36.633924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7956fdd4-6799-4d61-9651-f3c3f9af4d4d None None] MFA Rules not processed for user `9493f392c11542bcb36d37d3c8c18cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:36.636128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7956fdd4-6799-4d61-9651-f3c3f9af4d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:36.739246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1169/2344] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:36.749032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:36.750811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:36.803963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:36.817196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fedaf2-636b-4c7a-b4d0-e50cb0bc80e4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:36.820429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:36.821334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:36.821613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:36.821886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:36.883179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:36.884623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:36.894760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1176/2345] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:37.152097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:37.152471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:37.152927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:37.173777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:37.175121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:37.237071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1170/2346] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:49:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:37.250305 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:37.251742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:37.319904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07df7-ab1f-4253-9842-307c50c5b4ab service glance] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:37.325692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:37.326381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:37.326693 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:37.326968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:37.335728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:37.337311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973cfda-70b2-47bc-a31e-0b8182975684 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:37.348009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1177/2347] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:49:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:49:39.548355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:39.550443 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:39.646932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:39.748623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d111-d55b-4f80-a1fe-015b6655298a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:39.792496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:39.796620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:39.801006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:39.806993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:39.812533 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:39.844234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:39.861721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344074b-76e8-40e9-a043-7540abde9081 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:39.877637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1171/2348] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:43.762828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf3bf-7b9f-4f4c-9fe4-a0e5c1436721 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:43.763423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf3bf-7b9f-4f4c-9fe4-a0e5c1436721 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:43.763423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf3bf-7b9f-4f4c-9fe4-a0e5c1436721 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:43.764753 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1178/2349] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:49:43.776497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc22f-e427-42a1-91da-2cd20bdcdb09 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:43.776980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc22f-e427-42a1-91da-2cd20bdcdb09 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:43.777081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc22f-e427-42a1-91da-2cd20bdcdb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:43.830747 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b8cc22f-e427-42a1-91da-2cd20bdcdb09 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:43.841524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b8cc22f-e427-42a1-91da-2cd20bdcdb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:43.944232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1172/2350] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:49:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:43.958122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdf3bf-7b9f-4f4c-9fe4-a0e5c1436721 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:43.972515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdf3bf-7b9f-4f4c-9fe4-a0e5c1436721 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:43.977487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:43.978436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:43.979066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:43.979652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:43.995685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:43.997831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5261da-c1f6-4acc-9db3-66d5a3b55e54 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:44.022148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1179/2351] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:44.240203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.240973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.241444 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.243513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1173/2352] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:49:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:49:44.250839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeffc7-2c4c-47f9-a8a8-dbf57bf58013 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.251249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeffc7-2c4c-47f9-a8a8-dbf57bf58013 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.251622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeffc7-2c4c-47f9-a8a8-dbf57bf58013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.281169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cefeffc7-2c4c-47f9-a8a8-dbf57bf58013 None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:44.282902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cefeffc7-2c4c-47f9-a8a8-dbf57bf58013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:44.355598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1180/2353] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:49:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:49:44.363496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:44.365481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:44.382196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7b9e-17b6-4762-94c2-6c6d00138327 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:44.387698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:44.392784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.392784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.392784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.398104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:44.399682 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6ef8-d565-4961-8b74-945eba048af2 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:44.410414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1174/2354] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:44.667636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813ccab-c982-43ce-895f-6f8901bbea9c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.667636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813ccab-c982-43ce-895f-6f8901bbea9c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.667636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813ccab-c982-43ce-895f-6f8901bbea9c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.667636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1181/2355] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:49:44.670617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dcc91-6b9d-441d-90a3-1a04223fe090 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.671022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dcc91-6b9d-441d-90a3-1a04223fe090 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.671404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dcc91-6b9d-441d-90a3-1a04223fe090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.694766 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cf5dcc91-6b9d-441d-90a3-1a04223fe090 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:44.696090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf5dcc91-6b9d-441d-90a3-1a04223fe090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:44.714045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1175/2356] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:49:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:49:44.729812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813ccab-c982-43ce-895f-6f8901bbea9c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:44.764183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813ccab-c982-43ce-895f-6f8901bbea9c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:44.775149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:44.775956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:44.776332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:44.776702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:44.795095 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:44.796734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:44.811758 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1182/2357] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:49:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:49:50.373354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac441-8192-4c5a-be67-95d185a1e480 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:50.373904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac441-8192-4c5a-be67-95d185a1e480 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:50.373904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac441-8192-4c5a-be67-95d185a1e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:50.409340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1cbac441-8192-4c5a-be67-95d185a1e480 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:50.410850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1cbac441-8192-4c5a-be67-95d185a1e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:50.448299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1176/2358] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:49:50.463319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:49:50.481885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f184664-f6de-4c43-be90-4714e528cff9 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:49:50.488144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:49:50.489057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:50.489588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:50.490094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:50.498233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:49:50.500820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:49:50.503085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8f65d-ab89-4b87-a408-2332ef5c3e61 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:49:50.514065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1183/2359] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:49:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:49:52.134587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842bef5-b190-43ca-a400-139718e50990 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:49:52.135713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842bef5-b190-43ca-a400-139718e50990 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:49:52.136191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842bef5-b190-43ca-a400-139718e50990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:49:52.172690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f842bef5-b190-43ca-a400-139718e50990 None None] MFA Rules not processed for user `4ae0271efe4b4e9592b29d9f124519b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:49:52.174814 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f842bef5-b190-43ca-a400-139718e50990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:49:52.191337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1177/2360] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:49:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:00.564692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:02.162286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=65 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:04.758241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21b3b-c846-4a83-9dd8-cc28f52b09fc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:04.758775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21b3b-c846-4a83-9dd8-cc28f52b09fc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:04.758875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21b3b-c846-4a83-9dd8-cc28f52b09fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:04.779028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d8b21b3b-c846-4a83-9dd8-cc28f52b09fc None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:50:04.780338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d8b21b3b-c846-4a83-9dd8-cc28f52b09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:50:04.788191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1184/2361] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:50:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:04.801239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842bef5-b190-43ca-a400-139718e50990 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:50:04.815036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842bef5-b190-43ca-a400-139718e50990 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:50:04.820236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:50:04.820993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:04.821347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:04.821696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:04.832360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:50:04.834121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2b4f9-6f4e-48b8-b274-71f394a91ab5 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:50:04.844790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1178/2362] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:50:14.772885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:14.841155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:27.682329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:27.682914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:27.682914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:27.701586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] MFA Rules not processed for user `26ccc2d61c7045a8abba95e7cd002a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:50:27.703639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:50:27.773817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1185/2363] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:30.057063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:30.057537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:30.057685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:30.077865 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:50:30.079098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:50:30.088070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1179/2364] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 10:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:30.118709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:50:30.133233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a1f67-4168-4b67-91df-c3753603a7a5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:50:30.146330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:50:30.147076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:30.147426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:30.147768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:30.158401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:50:30.160051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ba657-49db-4eb7-96d2-014e87380257 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:50:30.168940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1186/2365] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:50:34.548871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:50:34.558301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ea335-759e-49ae-a35f-7a7a225845bd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:50:34.566059 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:50:34.566781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:34.567069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:34.567355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:34.630793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:50:34.632238 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:50:34.643154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1180/2366] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:50:40.165361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=31 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:44.633088 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1596-daae-4da4-bb6f-bafec015b26d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:44.634694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1596-daae-4da4-bb6f-bafec015b26d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:44.634694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1596-daae-4da4-bb6f-bafec015b26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:44.637995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=33 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:50:44.660311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-18fb1596-daae-4da4-bb6f-bafec015b26d None None] MFA Rules not processed for user `906e4bc4e9cc45ecb01cfca7918e461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:50:44.665636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-18fb1596-daae-4da4-bb6f-bafec015b26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:50:44.720499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1187/2367] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:48.855939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:50:48.868582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ba47c-0e85-4c37-97b8-d2caec8f2bc4 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:50:48.874211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:50:48.881555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:48.881555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:48.881555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:48.889643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:50:48.892170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:50:48.912196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1181/2368] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:50:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:50:52.169913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:50:52.170344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:50:52.170458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:50:52.216964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] MFA Rules not processed for user `a2cb1079722947dfa37946c03b7297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:50:52.219473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:50:52.348938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1188/2369] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:50:58.903604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:02.331832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=50 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:13.000575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:13.001951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:13.057846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4386c-5750-41d8-a3c1-64d4740de78b service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:13.061402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:13.062553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:13.062933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:13.063201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:13.065083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:13.079287 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:13.080705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:13.093125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1182/2370] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:16.978223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:16.987716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d03f7-d28d-42c6-b119-ef8540f464ed None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:16.991247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:16.991871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:16.992152 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:16.992427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:16.994189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:17.008468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:17.009889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f226-1c7f-4379-95e3-5287f0013cdd service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:17.021499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1189/2371] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:23.086897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=26 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:25.760572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:25.772317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f4b6-d7c8-42d0-b1d1-f7a8cb6c62f4 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:25.777824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:25.778374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:25.779189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:25.779468 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:25.781228 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:25.842326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:25.843943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:25.853601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1183/2372] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:27.015938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:35.849332 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=28 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:39.306571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:39.307115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:39.307115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:39.325372 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] MFA Rules not processed for user `23b63d44bb304e97a38acc6d0b791c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:51:39.326603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:39.334395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1190/2373] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:51:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:51:39.812717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:39.812717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069bf0-da70-4a22-93b0-cdfcffb61c8c service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:39.812717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:39.812717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:39.813470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:39.813470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:39.813470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:39.824397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:39.825825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:39.839688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1184/2374] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:51:40.325568 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:40.338233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17543ffe-3d0f-4b71-a33e-54a72496ce2b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:40.343626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:40.344280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:40.344564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:40.344841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:40.346673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:40.359245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:40.360824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:40.371340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1191/2375] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:43.143630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:43.155964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44aadb-d9e8-4f26-b0a0-9d6a3009868f service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:43.160655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:43.161832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:43.162286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:43.162757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:43.165442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:43.177396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:43.179022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:43.188016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1185/2376] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:47.938421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:47.941864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:48.014421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213c07-be94-4b41-a971-8d7fb0abda4d service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:48.019329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:48.020144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:48.020431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:48.020828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] PATH_INFO: `/v3/users/26ccc2d61c7045a8abba95e7cd002a0a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:48.022057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ccc2d61c7045a8abba95e7cd002a0a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:48.025110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:48.186480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ccc2d61c7045a8abba95e7cd002a0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.193624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ccc2d61c7045a8abba95e7cd002a0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.194321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ccc2d61c7045a8abba95e7cd002a0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.195591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ccc2d61c7045a8abba95e7cd002a0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.276201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ccc2d61c7045a8abba95e7cd002a0a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.294983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1192/2377] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:51:47 2022] DELETE /identity/v3/users/26ccc2d61c7045a8abba95e7cd002a0a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:51:48.631011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:48.644961 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82f6a5-f290-46c7-9bb9-a2bd55dbd47a service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:48.711776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:48.715019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:48.715019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:48.715019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] PATH_INFO: `/v3/projects/24d5a2477eb74d5da5911b37f9031110` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:48.715019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d5a2477eb74d5da5911b37f9031110)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:48.716649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:48.824257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d5a2477eb74d5da5911b37f9031110', 'request_id': 'req-9705953d-5ff7-4f36-91ff-6216876f49fb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.904488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d5a2477eb74d5da5911b37f9031110', 'request_id': 'req-9705953d-5ff7-4f36-91ff-6216876f49fb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.910669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d5a2477eb74d5da5911b37f9031110', 'request_id': 'req-9705953d-5ff7-4f36-91ff-6216876f49fb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.922173 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] The token cache is being invalidate because project 24d5a2477eb74d5da5911b37f9031110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:51:48.922461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.923118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:51:48.924642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1186/2378] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:51:48 2022] DELETE /identity/v3/projects/24d5a2477eb74d5da5911b37f9031110 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:51:51.066816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:51.068190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:51.130977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ac5a-a19d-4849-b92e-b7a7efd41feb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:51.143119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:51.182327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:51.182998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:51.183277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:51.183551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:51.185281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:51.245821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:51.247278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:51.256028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1193/2379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:54.324079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:54.325584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:54.424681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:51:54.430934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705953d-5ff7-4f36-91ff-6216876f49fb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:54.438833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:54.445365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:54.445365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:54.445365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:54.445365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:54.455744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:54.457390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:54.469449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1187/2380] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:55.548492 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:55.561641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ca6f3-3376-4ffc-8e51-4182ecee6c46 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:55.567049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:55.567682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:55.567965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:55.568242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:55.570027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:55.628836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:55.630434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:55.639779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1194/2381] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:51:59.003178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:51:59.020082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b711ec-be13-4b9c-8cc8-e2d4fa5be55d service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:51:59.024105 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:51:59.025336 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:51:59.025620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:51:59.025897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:51:59.027758 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:51:59.041799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:51:59.043843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:51:59.056114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1188/2382] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:05.636646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=69 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:09.050909 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=11 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:18.392662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:18.394144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:18.452382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517faa9-64bb-47ab-b3d4-81bf884b343c service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:18.456002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:18.459472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:18.459472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:18.459472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:18.459472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:18.522702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:18.524145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:18.535163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1195/2383] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:22.234716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:22.236658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:22.328251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24cd49-f745-46a3-90eb-ce6b88d72239 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:22.335174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:22.337567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:22.337567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:22.337567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:22.338471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:22.353043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:22.354520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:22.365685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1189/2384] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:52:22.396625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:22.398075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:22.420174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18884e1-9ab3-4ed6-bea5-a9ff73986def service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:22.420174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:22.420174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:22.420610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:22.420802 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] PATH_INFO: `/v3/users/a2cb1079722947dfa37946c03b7297a3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:22.421984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2cb1079722947dfa37946c03b7297a3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:22.423935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:22.661735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2cb1079722947dfa37946c03b7297a3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:22.668993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2cb1079722947dfa37946c03b7297a3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:22.669700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2cb1079722947dfa37946c03b7297a3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:22.670303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2cb1079722947dfa37946c03b7297a3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:22.754701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e3cd16c3-b973-48a6-91c0-e7b46c36bd47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2cb1079722947dfa37946c03b7297a3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:22.773288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1196/2385] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:52:22 2022] DELETE /identity/v3/users/a2cb1079722947dfa37946c03b7297a3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:23.196804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.209199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a349-5646-4c99-9953-334875bb0071 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.275759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.276684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.277023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.277507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] PATH_INFO: `/v3/projects/2ff45af5bd82457b9fd080026ea65b5b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.278770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff45af5bd82457b9fd080026ea65b5b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.280720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.345321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e8f0-ecb1-4a0d-87a0-0655db89c350 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.345612 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e8f0-ecb1-4a0d-87a0-0655db89c350 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.345887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e8f0-ecb1-4a0d-87a0-0655db89c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.364402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3a74e8f0-ecb1-4a0d-87a0-0655db89c350 None None] MFA Rules not processed for user `906e4bc4e9cc45ecb01cfca7918e461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:23.366000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3a74e8f0-ecb1-4a0d-87a0-0655db89c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:23.374990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff45af5bd82457b9fd080026ea65b5b', 'request_id': 'req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:23.383410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:23.420537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1197/2386] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:52:23.423312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff45af5bd82457b9fd080026ea65b5b', 'request_id': 'req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:23.429913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff45af5bd82457b9fd080026ea65b5b', 'request_id': 'req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:23.443881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] The token cache is being invalidate because project 2ff45af5bd82457b9fd080026ea65b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:52:23.444191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:23.444832 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-14c2ed18-4aaa-4cce-a8af-ef3814c6a187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:23.446479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1190/2387] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:52:23 2022] DELETE /identity/v3/projects/2ff45af5bd82457b9fd080026ea65b5b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:23.452746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.453036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.453889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.472808 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:23.474086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:23.536100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1198/2388] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:52:23.544299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.544590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.544873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.564175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:23.565378 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:23.573462 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1191/2389] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:23.577857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.587401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee47ea-37e2-4633-96d1-bac2ac8b222e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.593040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.593700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.593978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.594255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.595023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:52:23.595330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.596919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.609735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1199/2390] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:52:23.614152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.624301 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5a2ae-ed52-43ec-8cc7-d5c2f72f143f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.628977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.629630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.629913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.630188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.630927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.632858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.643038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:23.688441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1192/2391] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:52:23 2022] POST /identity/v3/projects => generated 403 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:52:23.694796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.705745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572637-a88e-45ad-96c8-50e60511d807 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.743166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.744877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:23.764775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.765474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.765794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.766074 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.766830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.768755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.804860 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19fb5c-de6c-455e-898b-b74461b30fad admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.809218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.810029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.810337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.810640 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.812408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:23.821654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.823112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.831991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1193/2392] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:52:23.866463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1200/2393] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:52:23 2022] POST /identity/v3/users => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:23.867841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.880157 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d01d0-54ed-4026-8f9b-f55af1ab4c34 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.884958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.885617 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.885924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.886200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.888577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.890428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.905903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1194/2394] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:52:23.907243 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.918858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fed341-43f1-464b-aad0-cb3568b603bf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.923728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.924363 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.924642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.924917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.925697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.927356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44da90d-6875-493f-b027-b9b3795ebc83 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:23.942228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1201/2395] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:23.943425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:23.954421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d8ef-f63e-4622-9380-f4620377ac2c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:23.959207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:23.960609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:23.960920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:23.961200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] PATH_INFO: `/v3/projects/23022ffa87534de39f3bad34c70d1d48/users/d952c741da6a475fa3c6b682de8ad970/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:23.977968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23022ffa87534de39f3bad34c70d1d48, user_id=d952c741da6a475fa3c6b682de8ad970, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:23.981317 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:24.059279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1195/2396] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:23 2022] PUT /identity/v3/projects/23022ffa87534de39f3bad34c70d1d48/users/d952c741da6a475fa3c6b682de8ad970/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:24.064554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72efb6-382f-46a5-8359-02f43f29d64a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:24.064868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72efb6-382f-46a5-8359-02f43f29d64a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:24.065475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72efb6-382f-46a5-8359-02f43f29d64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:24.083739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2b72efb6-382f-46a5-8359-02f43f29d64a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:24.091958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b72efb6-382f-46a5-8359-02f43f29d64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:24.115902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:24.146141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1202/2397] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:24.157072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:24.158874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:24.236410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c18d-428c-4fef-9451-16e689e3108e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:24.240543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:24.241298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:24.241681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:24.242056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:24.253011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:24.254592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:24.270942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1196/2398] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:25.675672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:25.684151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:25.684151 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:25.704118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] MFA Rules not processed for user `d952c741da6a475fa3c6b682de8ad970`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:25.705337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:25.771804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1203/2399] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:25.776473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:25.787693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcbd6-6a1f-4785-a86e-87d0c716eb1f service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:25.792358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:25.793051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:25.793502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:25.793788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:25.794560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:25.796465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:25.893791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1197/2400] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:52:25 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:25.897941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:25.909536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9841e-ff07-4f5c-9779-9e866531f4ae None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:25.968303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:25.968977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:25.969258 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:25.969533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:25.970213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:25.972166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:26.046444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1204/2401] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:52:25 2022] POST /identity/v3/users => generated 308 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:26.050484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:26.061628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad5d7-a4d6-40ab-991b-c13cb85e1e01 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:26.066297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:26.066993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:26.067320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:26.067581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:26.068302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:26.069995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:26.081501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1198/2402] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:26.085691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:26.095896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d423e4-6898-4bce-917e-66601746d1e6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:26.100710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:26.101581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:26.101860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:26.102135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] PATH_INFO: `/v3/projects/30e455e924de4b69ade561db9c1dcabb/users/d6cfacffe5234ec4a3c5c89f1a1b6d19/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:26.116933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e455e924de4b69ade561db9c1dcabb, user_id=d6cfacffe5234ec4a3c5c89f1a1b6d19, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:26.119567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:26.191066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1205/2403] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:26 2022] PUT /identity/v3/projects/30e455e924de4b69ade561db9c1dcabb/users/d6cfacffe5234ec4a3c5c89f1a1b6d19/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:26.194924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:26.205297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1daff7-606d-4480-8e79-432f9169c5a4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:26.232260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:26.264095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:26.264769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:26.265052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:26.265348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:26.266063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:26.267817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147f855-20a9-4634-b3a9-fb681198f1cd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:26.277704 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1199/2404] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:26.282054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:26.292401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371072b5-79d5-4be0-9b97-a5807e185901 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:26.298110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:26.299055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:26.299339 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:26.299619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] PATH_INFO: `/v3/projects/30e455e924de4b69ade561db9c1dcabb/users/d6cfacffe5234ec4a3c5c89f1a1b6d19/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:26.301747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e455e924de4b69ade561db9c1dcabb, user_id=d6cfacffe5234ec4a3c5c89f1a1b6d19, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:26.306415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:26.319526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1206/2405] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:26 2022] PUT /identity/v3/projects/30e455e924de4b69ade561db9c1dcabb/users/d6cfacffe5234ec4a3c5c89f1a1b6d19/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:27.716951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:27.717439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:27.717523 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:27.752500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] MFA Rules not processed for user `d6cfacffe5234ec4a3c5c89f1a1b6d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:27.753919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:27.833033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1200/2406] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:52:29.657839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:29.676397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:29.721208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9401a5-5d55-4a72-a08c-57ec92d0fef5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:29.725735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:29.726640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:29.726928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:29.727208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] PATH_INFO: `/v3/users/d952c741da6a475fa3c6b682de8ad970` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:29.728391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d952c741da6a475fa3c6b682de8ad970)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:29.730330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:29.912665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd952c741da6a475fa3c6b682de8ad970'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:29.920710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd952c741da6a475fa3c6b682de8ad970'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:29.921390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd952c741da6a475fa3c6b682de8ad970'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:29.921998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd952c741da6a475fa3c6b682de8ad970'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:29.999605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd952c741da6a475fa3c6b682de8ad970'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.021346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1207/2407] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:52:29 2022] DELETE /identity/v3/users/d952c741da6a475fa3c6b682de8ad970 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:30.258575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:30.270903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da0b6-be8f-4c36-87ee-cbc5163f506a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:30.330085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:30.330964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:30.331245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:30.331512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] PATH_INFO: `/v3/projects/23022ffa87534de39f3bad34c70d1d48` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:30.332676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23022ffa87534de39f3bad34c70d1d48)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:30.334571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:30.445956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23022ffa87534de39f3bad34c70d1d48', 'request_id': 'req-55e6e872-fa09-4161-876b-c0f62fb0bd39'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.448530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:30.548344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23022ffa87534de39f3bad34c70d1d48', 'request_id': 'req-55e6e872-fa09-4161-876b-c0f62fb0bd39'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.554083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23022ffa87534de39f3bad34c70d1d48', 'request_id': 'req-55e6e872-fa09-4161-876b-c0f62fb0bd39'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.565776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] The token cache is being invalidate because project 23022ffa87534de39f3bad34c70d1d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:52:30.566066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.566723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.568271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1201/2408] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:52:30 2022] DELETE /identity/v3/projects/23022ffa87534de39f3bad34c70d1d48 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:30.572314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:30.573759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:30.587066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90b9d9-e526-46f8-b72f-6d1f630367db admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:30.643303 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:30.644176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:30.644460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:30.644735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] PATH_INFO: `/v3/users/d6cfacffe5234ec4a3c5c89f1a1b6d19` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:30.645900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cfacffe5234ec4a3c5c89f1a1b6d19)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:30.647828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:30.822910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:30.831899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cfacffe5234ec4a3c5c89f1a1b6d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.838658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cfacffe5234ec4a3c5c89f1a1b6d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.839783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cfacffe5234ec4a3c5c89f1a1b6d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.841029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cfacffe5234ec4a3c5c89f1a1b6d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.885449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-18fec5b0-43d1-40ae-ba99-8089ba12d438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cfacffe5234ec4a3c5c89f1a1b6d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:30.904103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1208/2409] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:52:30 2022] DELETE /identity/v3/users/d6cfacffe5234ec4a3c5c89f1a1b6d19 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:31.133439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:31.135104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:31.152698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e872-fa09-4161-876b-c0f62fb0bd39 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:31.222429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:31.223323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.223605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.223878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] PATH_INFO: `/v3/projects/30e455e924de4b69ade561db9c1dcabb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.225075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e455e924de4b69ade561db9c1dcabb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:31.227026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:31.331994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e455e924de4b69ade561db9c1dcabb', 'request_id': 'req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:31.411670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e455e924de4b69ade561db9c1dcabb', 'request_id': 'req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:31.418552 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e455e924de4b69ade561db9c1dcabb', 'request_id': 'req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:31.430044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] The token cache is being invalidate because project 30e455e924de4b69ade561db9c1dcabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:52:31.430335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:31.431018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5dc0d7bc-ede4-453b-8f9c-a757e47c62d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:31.432935 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1202/2410] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:52:31 2022] DELETE /identity/v3/projects/30e455e924de4b69ade561db9c1dcabb => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:31.438651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.439010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.439294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.458107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:31.459474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:31.530734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1209/2411] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:31.532492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.532841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.533279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.552000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:31.553192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:31.562852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1203/2412] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:31.567169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:31.577608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9b5aa-abef-4155-94c6-c03e641a3ef1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:31.581939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:31.582662 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.582890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.583161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.583880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:52:31.584184 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:31.585771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:31.598448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1210/2413] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:52:31.602900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:31.613085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d400f8-cee3-4b45-aa27-a3567e007687 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:31.617875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:31.618610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.618897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.630098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.630098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:31.630098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:31.699313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1204/2414] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:52:31 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:52:31.709226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:31.737135 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f1f00-970b-4794-a4a8-863152df1d07 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:31.833076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:31.833898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.834188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.834468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.835207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:31.837160 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:31.957935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1211/2415] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:52:31 2022] POST /identity/v3/users => generated 287 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:31.961684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:31.976282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543afd99-1ab5-4238-a1f5-8efa8f0914f0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:31.983575 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:31.985123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:31.985435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:31.985717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:31.987348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:31.988980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.005249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1205/2416] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:32.008886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.021096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abe793-e58e-443e-9ea7-9e949ab32839 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.036002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.036819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.037232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.037232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.037955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.039664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3942-95bd-41bd-9752-2601dac61e8f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.055488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1212/2417] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:32.056786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.068098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7110c-f7cf-4935-b06a-f06d56cfea2f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.072850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.073921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.074296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.074707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] PATH_INFO: `/v3/projects/13d970de7fe443d3a2853c05f94c28f9/users/b8cb6d74e6834c8faecb2c6a5e9232a6/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.094977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d970de7fe443d3a2853c05f94c28f9, user_id=b8cb6d74e6834c8faecb2c6a5e9232a6, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.097755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.174881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1206/2418] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:32 2022] PUT /identity/v3/projects/13d970de7fe443d3a2853c05f94c28f9/users/b8cb6d74e6834c8faecb2c6a5e9232a6/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:32.186086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.186376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.186684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.221121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] MFA Rules not processed for user `b8cb6d74e6834c8faecb2c6a5e9232a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:32.222658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:32.233988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:32.311126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1213/2419] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:32.312440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.327587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:32.415798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af6dcd-e471-4128-9bb8-9a5ce82d2ce9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.422911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.423595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.423880 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.424158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.424857 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.426797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.522006 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1207/2420] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:52:32 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:32.525942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.539619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf8103-7f87-406f-b481-e38a4f27e4e0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.621173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.621173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.621173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.621173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.621643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.622131 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.705852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1214/2421] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:52:32 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:52:32.710944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.724079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b01d-b170-4a25-b14a-5042c0f386c9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.731514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.731514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.732132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.732479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.733289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.735044 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.751355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1208/2422] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:52:32.753520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.766270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f2d7-17bd-4cc3-98e7-15c6caace2fe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.776030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.778029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.778474 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.778787 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] PATH_INFO: `/v3/projects/32bbbd6677934da381797a18928e3087/users/2a791aeea68f4c8280cc72496b1a3a6c/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.806370 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bbbd6677934da381797a18928e3087, user_id=2a791aeea68f4c8280cc72496b1a3a6c, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.809180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:32.910084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1215/2423] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:32 2022] PUT /identity/v3/projects/32bbbd6677934da381797a18928e3087/users/2a791aeea68f4c8280cc72496b1a3a6c/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:52:32.914026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:32.927209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4017-28e1-4ba0-a822-be8a18df4713 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:32.990997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:32.991694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:32.992022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:32.992322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:32.993307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:32.994940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005034-9dd9-42a3-932a-758f7fb741a5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:33.008391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1209/2424] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:52:33.012795 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:33.024100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b8886-6028-4384-9e65-7db70c5ecdc7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:33.030569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:33.031487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:33.031764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:33.032037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] PATH_INFO: `/v3/projects/32bbbd6677934da381797a18928e3087/users/2a791aeea68f4c8280cc72496b1a3a6c/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:33.034255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bbbd6677934da381797a18928e3087, user_id=2a791aeea68f4c8280cc72496b1a3a6c, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:33.037071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:33.048752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1216/2425] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:52:33 2022] PUT /identity/v3/projects/32bbbd6677934da381797a18928e3087/users/2a791aeea68f4c8280cc72496b1a3a6c/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:33.059145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:33.059430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:33.059705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:33.087246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] MFA Rules not processed for user `2a791aeea68f4c8280cc72496b1a3a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:33.088505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:33.157475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1210/2426] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:52:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:33.179126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:33.181124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:33.240705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1a50-f6b1-4934-ad2e-fd1fce6f0d88 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:33.244386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:33.245028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:33.245308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:33.245611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:33.255053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:33.256468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:33.266591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1217/2427] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:33.469686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:33.474753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:33.481587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d260-00ce-42f9-adb9-e13070620664 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:33.485594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:33.486370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:33.487797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:33.487797 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:33.554207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:33.555789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab4b8-f319-4a1e-8489-c39674b275ad service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:33.566594 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1211/2428] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:34.510412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:34.512726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:34.519373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:34.590333 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3ac6-cdee-48d7-ba1a-e3d624ccf7aa service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:34.596155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:34.597238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:34.597569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:34.597847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:34.600462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:34.614812 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:34.616382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:34.641820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1218/2429] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:43.540861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:43.541271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:43.541410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:43.558636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:43.559819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:43.614878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1212/2430] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 10:52:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:43.752743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:43.754247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:43.822332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867d11f-2687-4d57-b37c-381ebe3d4d22 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:43.828056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:43.828691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:43.828972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:43.829246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:43.837657 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:43.839589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:43.848330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1219/2431] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:52:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:52:48.869708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:48.871729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:48.942627 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc78d1-fd51-4b70-8c08-eee5452cc62f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:48.947566 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:48.948209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:48.948497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:48.948775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:48.951082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:48.962414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:48.964319 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:48.974937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1213/2432] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:52:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:52:53.034872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:53.038644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:53.117032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a142e-47c9-4c8e-9da3-848c35e63e04 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:53.122847 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:53.124165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:53.127896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:53.127896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] PATH_INFO: `/v3/users/4ae0271efe4b4e9592b29d9f124519b7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:53.127896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae0271efe4b4e9592b29d9f124519b7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:53.128440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:53.152744 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:53.298191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae0271efe4b4e9592b29d9f124519b7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:53.304834 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae0271efe4b4e9592b29d9f124519b7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:53.305507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae0271efe4b4e9592b29d9f124519b7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:53.306168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae0271efe4b4e9592b29d9f124519b7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:53.394724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae0271efe4b4e9592b29d9f124519b7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:53.412909 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1220/2433] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:52:53 2022] DELETE /identity/v3/users/4ae0271efe4b4e9592b29d9f124519b7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:53.697711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:53.708960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa52f-90be-47ee-a8fa-4e137d008cff service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:53.746033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:53.770916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:53.771791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:53.772075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:53.772354 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] PATH_INFO: `/v3/projects/2eff05785bd84522ad637d8bed14563d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:53.773539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eff05785bd84522ad637d8bed14563d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:53.775585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:53.892421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eff05785bd84522ad637d8bed14563d', 'request_id': 'req-b89960f2-98df-486e-b0d0-9f6ff83c727b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.017386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eff05785bd84522ad637d8bed14563d', 'request_id': 'req-b89960f2-98df-486e-b0d0-9f6ff83c727b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.024027 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eff05785bd84522ad637d8bed14563d', 'request_id': 'req-b89960f2-98df-486e-b0d0-9f6ff83c727b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.035729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] The token cache is being invalidate because project 2eff05785bd84522ad637d8bed14563d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:52:54.036018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.036679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.038308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1214/2434] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:52:53 2022] DELETE /identity/v3/projects/2eff05785bd84522ad637d8bed14563d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:54.042249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:54.043711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:54.057598 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0987c25-7794-4790-ac1e-45251693bb4a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:54.115475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:54.116343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:54.116627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:54.116906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] PATH_INFO: `/v3/users/9493f392c11542bcb36d37d3c8c18cbb` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:54.118136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9493f392c11542bcb36d37d3c8c18cbb)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:54.120097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:54.319379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9493f392c11542bcb36d37d3c8c18cbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.327564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9493f392c11542bcb36d37d3c8c18cbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.328268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9493f392c11542bcb36d37d3c8c18cbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.328851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9493f392c11542bcb36d37d3c8c18cbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.412379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba365abd-e080-4e6b-8cab-ba05f96ec408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9493f392c11542bcb36d37d3c8c18cbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:54.432071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1221/2435] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:52:54 2022] DELETE /identity/v3/users/9493f392c11542bcb36d37d3c8c18cbb => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:54.715430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:54.718098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:54.731095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89960f2-98df-486e-b0d0-9f6ff83c727b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:54.791845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:54.792638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:54.792921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:54.793197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] PATH_INFO: `/v3/projects/2b05809a5167425aaee1fe39efa5fd4d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:54.794375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b05809a5167425aaee1fe39efa5fd4d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:54.796510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:54.942882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b05809a5167425aaee1fe39efa5fd4d', 'request_id': 'req-c85dba8c-9bdd-45db-9e89-c3712831617b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:55.021716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b05809a5167425aaee1fe39efa5fd4d', 'request_id': 'req-c85dba8c-9bdd-45db-9e89-c3712831617b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:55.027572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b05809a5167425aaee1fe39efa5fd4d', 'request_id': 'req-c85dba8c-9bdd-45db-9e89-c3712831617b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:55.030016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:55.040249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] The token cache is being invalidate because project 2b05809a5167425aaee1fe39efa5fd4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:52:55.040540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:55.041187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:52:55.043131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1215/2436] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:52:54 2022] DELETE /identity/v3/projects/2b05809a5167425aaee1fe39efa5fd4d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:52:56.093672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db2ed-0fb2-43f0-90fc-64bc36e7c40b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:56.094215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db2ed-0fb2-43f0-90fc-64bc36e7c40b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:56.094215 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db2ed-0fb2-43f0-90fc-64bc36e7c40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:56.112591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0c8db2ed-0fb2-43f0-90fc-64bc36e7c40b None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:52:56.113907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0c8db2ed-0fb2-43f0-90fc-64bc36e7c40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:56.126824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:52:56.172174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1222/2437] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:52:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:52:56.184265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:52:56.186379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:52:56.258608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dba8c-9bdd-45db-9e89-c3712831617b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:52:56.263973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:52:56.264226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:52:56.264507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:52:56.264780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:52:56.273236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:52:56.274845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c4b81-a44d-4f26-ac53-51d82a5941df service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:52:56.284043 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1216/2438] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:52:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:53:06.204329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=17 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:06.280204 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=34 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:13.299094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:13.299651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:13.299651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:13.301939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1223/2439] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:53:13.309302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed4e0-201e-4189-b209-ea6de9c7bb03 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:13.310145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed4e0-201e-4189-b209-ea6de9c7bb03 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:13.310433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed4e0-201e-4189-b209-ea6de9c7bb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:13.333225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f6aed4e0-201e-4189-b209-ea6de9c7bb03 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:13.333225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f6aed4e0-201e-4189-b209-ea6de9c7bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:13.398416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1217/2440] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:53:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:13.410673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:13.412365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:13.475956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8e493-d106-48ff-bd57-4136b65d8851 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:13.479696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:13.480462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:13.480836 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:13.481207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:13.499347 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:13.501544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dc0b3-4e5c-4742-94f0-80bc70f9fc41 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:13.513289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1224/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:13.769307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:13.769674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:13.769929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:13.771447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1218/2442] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:53:13.776597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86e7-6631-4c27-bb49-cf8aa87458e1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:13.776886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86e7-6631-4c27-bb49-cf8aa87458e1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:13.777166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86e7-6631-4c27-bb49-cf8aa87458e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:13.919933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-16ab86e7-6631-4c27-bb49-cf8aa87458e1 None None] MFA Rules not processed for user `dbd2ab88b240473aa0067856ec2254f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:13.921126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-16ab86e7-6631-4c27-bb49-cf8aa87458e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:13.975149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1225/2443] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:53:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:13.988082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:13.989895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:14.003830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194549e-82da-422a-afd3-5ace19e9d298 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:14.007624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:14.008265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:14.008545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:14.008817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:14.017000 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:14.018412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:14.027152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1219/2444] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:20.575125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aac1-7a04-4772-ac68-92dc3ee9fe69 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:20.575681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aac1-7a04-4772-ac68-92dc3ee9fe69 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:20.575790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aac1-7a04-4772-ac68-92dc3ee9fe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:20.595421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8b84aac1-7a04-4772-ac68-92dc3ee9fe69 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:20.596627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8b84aac1-7a04-4772-ac68-92dc3ee9fe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:20.604939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1226/2445] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:53:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:20.619979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:20.635302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902b33-8820-4e3e-873c-34988e081d4f service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:20.641097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:20.642751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:20.643210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:20.643649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:20.655009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:20.657358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67a1a4-3892-4d73-a677-95bdfed3c804 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:20.677789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1220/2446] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:53:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:53:21.340070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac38c-b3fe-4187-86f3-c2fba281ec76 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:21.340751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac38c-b3fe-4187-86f3-c2fba281ec76 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:21.341031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac38c-b3fe-4187-86f3-c2fba281ec76 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:21.342826 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1227/2447] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 21 10:53:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 10:53:21.350037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:21.350691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:21.351247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:21.397750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] MFA Rules not processed for user `23fabbe0e78d45f890b21fcf4b348721`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:21.399030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:21.425071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1221/2448] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 21 10:53:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:21.441485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ac38c-b3fe-4187-86f3-c2fba281ec76 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:21.460025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ac38c-b3fe-4187-86f3-c2fba281ec76 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:21.464548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:21.465577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:21.466048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:21.466719 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:21.480413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:21.482319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe39fd-5e3e-4619-b14c-ab97ad5337b5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:21.507914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1228/2449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:23.277974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:23.279402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:23.349198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d730-07bf-4401-9ac3-1d235e479d09 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:23.354341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:23.355455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:23.355885 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:23.356368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] PATH_INFO: `/v3/users/b8cb6d74e6834c8faecb2c6a5e9232a6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:23.357981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cb6d74e6834c8faecb2c6a5e9232a6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:23.360107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:23.546517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cb6d74e6834c8faecb2c6a5e9232a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:23.554500 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cb6d74e6834c8faecb2c6a5e9232a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:23.558516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:23.673265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cb6d74e6834c8faecb2c6a5e9232a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:23.674024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cb6d74e6834c8faecb2c6a5e9232a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:23.674759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cb6d74e6834c8faecb2c6a5e9232a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:23.695306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1222/2450] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:23 2022] DELETE /identity/v3/users/b8cb6d74e6834c8faecb2c6a5e9232a6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:23.699630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:23.700162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:23.700458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:23.722385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:23.723848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:23.826272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1229/2451] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:23.842744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:23.844460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:23.912944 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02564-5f77-4f1c-ae7a-4fea0629ad9c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:23.918073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:23.918752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:23.919038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:23.919392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:23.929769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:23.931196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:23.950811 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1223/2452] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:24.423029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:24.424902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:24.440771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb193aa-7d18-498c-b366-97d7c0f9352e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:24.445951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:24.447079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:24.447904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:24.448355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] PATH_INFO: `/v3/projects/13d970de7fe443d3a2853c05f94c28f9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:24.453374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d970de7fe443d3a2853c05f94c28f9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:24.453374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:24.563038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:24.581450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d970de7fe443d3a2853c05f94c28f9', 'request_id': 'req-68edb39b-cdb4-4618-b4d0-3d07392d41ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:24.672862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d970de7fe443d3a2853c05f94c28f9', 'request_id': 'req-68edb39b-cdb4-4618-b4d0-3d07392d41ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:24.681560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d970de7fe443d3a2853c05f94c28f9', 'request_id': 'req-68edb39b-cdb4-4618-b4d0-3d07392d41ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:24.692538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] The token cache is being invalidate because project 13d970de7fe443d3a2853c05f94c28f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:24.692823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:24.694132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:24.699239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1230/2453] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:24 2022] DELETE /identity/v3/projects/13d970de7fe443d3a2853c05f94c28f9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:24.700475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:24.716284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c3fdf-7c37-4cf4-977b-2c84da8fb5d2 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:24.778110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:24.779136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:24.779426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:24.779702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] PATH_INFO: `/v3/users/2a791aeea68f4c8280cc72496b1a3a6c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:24.780923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a791aeea68f4c8280cc72496b1a3a6c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:24.783496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:25.039087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a791aeea68f4c8280cc72496b1a3a6c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:25.046977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a791aeea68f4c8280cc72496b1a3a6c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:25.124587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a791aeea68f4c8280cc72496b1a3a6c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:25.125296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a791aeea68f4c8280cc72496b1a3a6c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:25.125992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6ee2ac0d-adf9-4f51-857a-a513bcded67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a791aeea68f4c8280cc72496b1a3a6c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:25.141133 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:25.145490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1224/2454] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:24 2022] DELETE /identity/v3/users/2a791aeea68f4c8280cc72496b1a3a6c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:25.699729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:25.701394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:25.719000 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb39b-cdb4-4618-b4d0-3d07392d41ab admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:25.794192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:25.796329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:25.796822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:25.797267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] PATH_INFO: `/v3/projects/32bbbd6677934da381797a18928e3087` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:25.798780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bbbd6677934da381797a18928e3087)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:25.800695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:25.940855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bbbd6677934da381797a18928e3087', 'request_id': 'req-04f0fa9e-1195-4ae9-a005-914a8858b3ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:26.010044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bbbd6677934da381797a18928e3087', 'request_id': 'req-04f0fa9e-1195-4ae9-a005-914a8858b3ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:26.019092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bbbd6677934da381797a18928e3087', 'request_id': 'req-04f0fa9e-1195-4ae9-a005-914a8858b3ab'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:26.035723 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] The token cache is being invalidate because project 32bbbd6677934da381797a18928e3087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:26.036093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:26.036672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-04f0fa9e-1195-4ae9-a005-914a8858b3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:26.041588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1231/2455] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:25 2022] DELETE /identity/v3/projects/32bbbd6677934da381797a18928e3087 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:26.043295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.043748 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.043954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.064550 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:26.066060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:26.153383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1225/2456] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:26.166450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.166769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.167066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.201531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:26.203009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:26.216234 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1232/2457] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:26.217548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.230676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b01f4-5ad5-40fe-9949-fba1b763e9b5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.236619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.237700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.238144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.238701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.239604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:53:26.239920 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.242726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.256096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1226/2458] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:53:26.261098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.271486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6275c35-3f4b-4df5-9a43-7657ac265741 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.276431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.277358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.277640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.277914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.278663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.280589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.384111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1233/2459] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:26.388590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.399997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae92028-74ee-41a6-b0b6-968f0d7ca9ec admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.486040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.486799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.487083 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.487396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.488062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.490340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.577424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1227/2460] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:26.581846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.600413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693f4f8-c79a-4bdd-a3c1-a29fc8584c03 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.610508 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.612108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.612656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.613207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.614267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.616307 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.628336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1234/2461] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:26.632540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.643844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c225ed9-2a28-4477-85ab-5ebbfbd4cb59 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.649190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.649938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.650293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.650658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.651652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.654174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58314b3d-295d-4fbc-a96e-72176f1df177 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.670890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1228/2462] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:26.675154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:26.692487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595fa2-bd75-4e9c-950e-b351968d13a9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:26.701645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:26.702570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.702860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.703137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] PATH_INFO: `/v3/projects/e6b9664853754432a0ef89bd4bfe1bfe/users/79b0cdee23144b88a0e1d1b27d1d6e77/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.706384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:26.731285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b9664853754432a0ef89bd4bfe1bfe, user_id=79b0cdee23144b88a0e1d1b27d1d6e77, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:26.734353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:26.852844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1235/2463] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:26 2022] PUT /identity/v3/projects/e6b9664853754432a0ef89bd4bfe1bfe/users/79b0cdee23144b88a0e1d1b27d1d6e77/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:26.863762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:26.864050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:26.864327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:26.894449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] MFA Rules not processed for user `79b0cdee23144b88a0e1d1b27d1d6e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:26.900273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:26.968095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:26.984141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1229/2464] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:26.984893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.064346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf78be-d3ca-4496-a3a9-ea09a64b0c72 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.070694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.071327 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.071666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.071950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.072655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.074569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:27.161513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1236/2465] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:26 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:27.165614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.178236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccd1b4-2456-440c-9a46-1ab3c4d5dbf0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.266186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.267285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.267686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.268042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.268818 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.270867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:27.365458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1230/2466] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:27 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:27.366283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.382991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce5dcd-fdbd-44bf-bac9-0cde8430324c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.396432 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.397445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.397849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.398223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.399089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.400775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:27.415279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1237/2467] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:27.419453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.430321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e34fb-0525-46f1-8e0d-c81c836413ae admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.435130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.435804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.436089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.436369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.437346 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.439772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce4492-0b58-48fe-a89e-cbc85ddfd1a9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:27.451342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1231/2468] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:27.456438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.471731 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ba79-ec3c-4aaa-93c9-9af9efb3ce97 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.478446 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.479410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.479733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.480039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] PATH_INFO: `/v3/projects/9f8d0eed14814ddb9c43028d5e77db78/users/a91dc4a1e8614e13be81e2645753234e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.500162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8d0eed14814ddb9c43028d5e77db78, user_id=a91dc4a1e8614e13be81e2645753234e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.503962 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:27.743362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1238/2469] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:27 2022] PUT /identity/v3/projects/9f8d0eed14814ddb9c43028d5e77db78/users/a91dc4a1e8614e13be81e2645753234e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:27.756047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.759525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.759525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.791206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] MFA Rules not processed for user `a91dc4a1e8614e13be81e2645753234e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:27.792535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:27.879932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1232/2470] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:27.884611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:27.960822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a127d-8a0e-44ca-83a2-12cd5ca00adc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:27.966398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:27.967176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:27.967457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:27.967736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:27.968517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:27.971313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.058314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1239/2471] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:27 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:28.062726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.082802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0073b7c-1730-4c0b-ac79-5d96ee189cd0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.155908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.156585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.156872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.157152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.158558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.165168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.170248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:28.257822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1233/2472] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:28 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:28.261471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.276352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76bab-fd2b-4455-a59b-3cb03b7fb181 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.283626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.284844 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.285229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.285627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.291582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.293162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.305128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1240/2473] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:28.309506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.320700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1931-986b-4a80-90e5-cff67d038a64 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.325445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.326341 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.326659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.326942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] PATH_INFO: `/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3/users/1369019dcae44d58bd02191c3f2cc631/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.344164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d2e95db57241a5ac7c19d2dfb68da3, user_id=1369019dcae44d58bd02191c3f2cc631, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.347586 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.428162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1234/2474] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:28 2022] PUT /identity/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3/users/1369019dcae44d58bd02191c3f2cc631/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:28.432444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.443835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8efa-6f1d-45f8-a4f8-836c155c4f04 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.447524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:28.505767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.506528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.506918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.507196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.507924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.509520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b057d09-6483-4817-8a4d-6214b590010a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.524472 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1241/2475] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:28.525379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.539697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd035c3-a4a0-4606-9ec5-fcd47b1f0629 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.545040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.545915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.546345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.547022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] PATH_INFO: `/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3/users/1369019dcae44d58bd02191c3f2cc631/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.549303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d2e95db57241a5ac7c19d2dfb68da3, user_id=1369019dcae44d58bd02191c3f2cc631, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.552203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.564109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1235/2476] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:28 2022] PUT /identity/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3/users/1369019dcae44d58bd02191c3f2cc631/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:28.574768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.575059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.575332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.601530 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] MFA Rules not processed for user `1369019dcae44d58bd02191c3f2cc631`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:28.602748 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:28.670450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1242/2477] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:28.681934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:28.683644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:28.741038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2b3-481e-414c-9d05-b01497ac9b95 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:28.744604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:28.745234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:28.745512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:28.745809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:28.755055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:28.756478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222352b-87dd-4eea-8057-fc57772c4fe9 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:28.765571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1236/2478] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:29.899005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:29.909761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cffbf5-f861-4fe1-99bc-af068af9c3d2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:29.913348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:29.913979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:29.914261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:29.914552 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:29.971493 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:29.972943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:29.984911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1243/2479] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:34.650212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:34.653247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:34.653247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:34.669725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:34.670963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:34.729134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1237/2480] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 10:53:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:34.745736 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:34.747500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:34.784505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:34.828182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381543cb-5c7e-4fa1-94ac-0c71ec7e24a3 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:34.839777 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:34.840522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:34.840881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:34.841228 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:34.853296 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:34.855273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:34.866463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1244/2481] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:53:36.184565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:36.197394 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a108970-b18d-43ef-bc6b-7008557f3af2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:36.201114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:36.201752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:36.202032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:36.202309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:36.266125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:36.267686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547c00-ffe0-43ca-b8b8-1b4d3a326cc3 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:36.277158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1238/2482] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:40.244975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:40.311023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dcde-acb0-4ae3-96be-f38af499ad4b service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:40.316516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:40.317385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:40.317661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:40.317937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] PATH_INFO: `/v3/users/79b0cdee23144b88a0e1d1b27d1d6e77` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:40.319418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b0cdee23144b88a0e1d1b27d1d6e77)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:40.321310 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:40.520546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b0cdee23144b88a0e1d1b27d1d6e77'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:40.528222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b0cdee23144b88a0e1d1b27d1d6e77'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:40.529203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b0cdee23144b88a0e1d1b27d1d6e77'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:40.529610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b0cdee23144b88a0e1d1b27d1d6e77'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:40.609499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b0cdee23144b88a0e1d1b27d1d6e77'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:40.628274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1245/2483] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:40 2022] DELETE /identity/v3/users/79b0cdee23144b88a0e1d1b27d1d6e77 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:40.633226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:40.633808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:40.633808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:40.639792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:40.653344 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:40.654535 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:40.713123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1239/2484] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:40.722857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:40.724313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:40.764060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:40.780814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e58fc-ac84-40ac-ace0-7f1d85b2b995 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:40.784522 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:40.785164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:40.785490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:40.785727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:40.794653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:40.796114 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:40.806197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1246/2485] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:41.274046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:41.275683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:41.288445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471ef30-935c-44dc-b443-0c422338f983 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:41.293160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:41.294085 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:41.294431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:41.294791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] PATH_INFO: `/v3/projects/e6b9664853754432a0ef89bd4bfe1bfe` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:41.296029 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b9664853754432a0ef89bd4bfe1bfe)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:41.297932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:41.419793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b9664853754432a0ef89bd4bfe1bfe', 'request_id': 'req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.524295 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b9664853754432a0ef89bd4bfe1bfe', 'request_id': 'req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.531639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b9664853754432a0ef89bd4bfe1bfe', 'request_id': 'req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.544037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] The token cache is being invalidate because project e6b9664853754432a0ef89bd4bfe1bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:41.544374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.545073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.549954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1240/2486] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:41 2022] DELETE /identity/v3/projects/e6b9664853754432a0ef89bd4bfe1bfe => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:41.551532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:41.561211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ece5a-4cf3-428d-bf1f-e62abe62b73b service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:41.620540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:41.621334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:41.621608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:41.621876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] PATH_INFO: `/v3/users/a91dc4a1e8614e13be81e2645753234e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:41.623099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91dc4a1e8614e13be81e2645753234e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:41.625022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:41.776845 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91dc4a1e8614e13be81e2645753234e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.785060 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91dc4a1e8614e13be81e2645753234e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.785712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91dc4a1e8614e13be81e2645753234e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.786304 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91dc4a1e8614e13be81e2645753234e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.871512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91dc4a1e8614e13be81e2645753234e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:41.891591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1247/2487] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:41 2022] DELETE /identity/v3/users/a91dc4a1e8614e13be81e2645753234e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:42.325971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:42.327870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:42.342400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fc2a5-257f-4969-a529-ef5e6d5e1d55 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:42.402372 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:42.403185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:42.403467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:42.403740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] PATH_INFO: `/v3/projects/9f8d0eed14814ddb9c43028d5e77db78` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:42.404911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8d0eed14814ddb9c43028d5e77db78)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:42.406839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:42.492932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:42.509135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8d0eed14814ddb9c43028d5e77db78', 'request_id': 'req-586a26d1-d01c-44d5-bbb0-658ba13332a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.597272 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8d0eed14814ddb9c43028d5e77db78', 'request_id': 'req-586a26d1-d01c-44d5-bbb0-658ba13332a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.602964 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8d0eed14814ddb9c43028d5e77db78', 'request_id': 'req-586a26d1-d01c-44d5-bbb0-658ba13332a2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.618246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] The token cache is being invalidate because project 9f8d0eed14814ddb9c43028d5e77db78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:42.618705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.619423 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.621276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1241/2488] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:42 2022] DELETE /identity/v3/projects/9f8d0eed14814ddb9c43028d5e77db78 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:42.625393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:42.626906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:42.639999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c2f02-f3b7-4b8e-ad3a-9dd39dd2e7ca admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:42.710743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:42.717672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:42.718569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:42.718992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:42.719274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] PATH_INFO: `/v3/users/1369019dcae44d58bd02191c3f2cc631` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:42.720712 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1369019dcae44d58bd02191c3f2cc631)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:42.722904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:42.899349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1369019dcae44d58bd02191c3f2cc631'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.906454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1369019dcae44d58bd02191c3f2cc631'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.907155 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1369019dcae44d58bd02191c3f2cc631'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.907766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1369019dcae44d58bd02191c3f2cc631'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.969817 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4e64ffd4-0f03-47ff-b6f6-8ce7e8c674fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1369019dcae44d58bd02191c3f2cc631'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:42.988949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1248/2489] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:42 2022] DELETE /identity/v3/users/1369019dcae44d58bd02191c3f2cc631 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:43.437815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:43.440275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:43.454055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a26d1-d01c-44d5-bbb0-658ba13332a2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:43.525785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:43.526685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:43.526968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:43.527245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] PATH_INFO: `/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:43.533213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d2e95db57241a5ac7c19d2dfb68da3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:43.533213 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:43.630382 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d2e95db57241a5ac7c19d2dfb68da3', 'request_id': 'req-711b0e5b-55cd-4165-9448-082e19cde812'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:43.695962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d2e95db57241a5ac7c19d2dfb68da3', 'request_id': 'req-711b0e5b-55cd-4165-9448-082e19cde812'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:43.703347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d2e95db57241a5ac7c19d2dfb68da3', 'request_id': 'req-711b0e5b-55cd-4165-9448-082e19cde812'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:43.715760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] The token cache is being invalidate because project c7d2e95db57241a5ac7c19d2dfb68da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:43.716048 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:43.716722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-711b0e5b-55cd-4165-9448-082e19cde812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:43.718320 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1242/2490] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:43 2022] DELETE /identity/v3/projects/c7d2e95db57241a5ac7c19d2dfb68da3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:43.725028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:43.725519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:43.725615 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:43.750703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:43.752605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:43.836410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1249/2491] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:43.848447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:43.848733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:43.849017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:43.872868 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:43.874509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:43.885526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1243/2492] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:43.889597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:43.902412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90092-995b-40a2-a7fb-4f7000a94557 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:43.907500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:43.909230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:43.909531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:43.909914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:43.910732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:53:43.911241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:43.913026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:43.926832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1250/2493] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:53:43.933510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:43.944691 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c388b7-8b88-49d3-90bf-2ad92d3367d9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:43.949770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:43.950504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:43.950894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:43.951293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:43.951994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:43.955396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.055794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1244/2494] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:43 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:44.057596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.068822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162163d-2963-4f69-93fc-2145f79353e3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.133821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.134482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.134791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.135066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.135739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.137636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.146219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:44.227222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1251/2495] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:44 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:44.231646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.242846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6d07-f2cb-4a70-8019-066b916a83b0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.247473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.248224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.248505 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.248779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.249488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.251099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.262939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1245/2496] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:44.267230 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.277622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c10e4-84ce-451d-9089-226d3ce32788 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.282189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.282871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.283285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.283565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.284285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.285872 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54f4a2-4b89-4e94-b137-15ae0c5eb3e1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.296649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1252/2497] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:44.301712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.312042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8155-117d-49e3-9668-9c0cf305ab61 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.317618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.318605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.318892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.319168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] PATH_INFO: `/v3/projects/802ab49587934150af9e61c70d2fc06b/users/8ccea90a4c5c4fdeb7e53af55f102a5b/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.328016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:44.335490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802ab49587934150af9e61c70d2fc06b, user_id=8ccea90a4c5c4fdeb7e53af55f102a5b, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.338681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.456153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1246/2498] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:44 2022] PUT /identity/v3/projects/802ab49587934150af9e61c70d2fc06b/users/8ccea90a4c5c4fdeb7e53af55f102a5b/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:44.468051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.468342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.468621 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.498606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] MFA Rules not processed for user `8ccea90a4c5c4fdeb7e53af55f102a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:44.499814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:44.566832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1253/2499] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:44.571299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.658974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53066-9e29-4721-b87c-f0f9ca113de5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.663834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.664580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.665050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.665403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.666490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.668574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.772960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1247/2500] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:44 2022] POST /identity/v3/projects => generated 391 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:44.776729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.789051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8db606-692d-4cdf-86f8-529b76c2aedf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.850702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.851292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.851577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.851856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.852561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.854534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:44.972562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1254/2501] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:44 2022] POST /identity/v3/users => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:44.976720 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:44.988358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f187336-be9a-4e15-94c3-863a1e9b6198 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:44.993576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:44.994763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:44.994763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:44.994986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:44.996404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:44.997527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.011310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1248/2502] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:45.015593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:45.027785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b68c6-f58c-4c55-98e0-c106e9a26c3c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:45.032880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:45.035065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.035341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.035611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] PATH_INFO: `/v3/projects/c057f0f966bb47738340dbbc2d60b19a/users/37f4822be380498e993a34d6464cc216/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.052008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c057f0f966bb47738340dbbc2d60b19a, user_id=37f4822be380498e993a34d6464cc216, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:45.054761 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.148065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1255/2503] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:45 2022] PUT /identity/v3/projects/c057f0f966bb47738340dbbc2d60b19a/users/37f4822be380498e993a34d6464cc216/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:45.152047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:45.162913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023ee8e-6f34-4ab1-aa3e-c67c308290ba admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:45.224572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:45.225237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.225519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.225793 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.226513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:45.228125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cb508-7224-4e80-8696-cce692ce5910 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.238211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1249/2504] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:45.242592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:45.254311 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce684af2-6aca-42ad-b69c-ad34b19b34be admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:45.259087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:45.259949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.260274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.260550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] PATH_INFO: `/v3/projects/c057f0f966bb47738340dbbc2d60b19a/users/37f4822be380498e993a34d6464cc216/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.262586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c057f0f966bb47738340dbbc2d60b19a, user_id=37f4822be380498e993a34d6464cc216, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:45.265403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.277029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1256/2505] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:45 2022] PUT /identity/v3/projects/c057f0f966bb47738340dbbc2d60b19a/users/37f4822be380498e993a34d6464cc216/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:45.288069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.288360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.288641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.315126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] MFA Rules not processed for user `37f4822be380498e993a34d6464cc216`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:45.316322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:45.357504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:45.384412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1250/2506] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:45.395997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:45.397721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:45.425805 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:45.455914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467c82a-0f9d-4353-aecc-ee837f0d161a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:45.459694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:45.460332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.460648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.460925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.470094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:45.471645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.481356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1257/2507] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:45.528390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:45.529772 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:45.541656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a71d-65fd-4e4c-8311-0239db40caf8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:45.545236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:45.545864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:45.546140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:45.546414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:45.603451 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:45.604956 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:45.615116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1251/2508] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:46.121467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:46.123971 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:46.197796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2068-386c-46f6-a60d-f14786d2977c service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:46.201724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:46.205999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:46.205999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:46.205999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:46.205999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:46.290981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:46.292500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:46.303280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1258/2509] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:50.310086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:50.311697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:50.329172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf80dc-5e70-447d-b335-dbe35e71a45e service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:50.334779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:50.335655 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:50.335929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:50.336218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] PATH_INFO: `/v3/users/906e4bc4e9cc45ecb01cfca7918e461d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:50.337786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=906e4bc4e9cc45ecb01cfca7918e461d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:50.341432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:50.411884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:50.421498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6e74-613a-4dee-bf37-66aa11c52028 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:50.426762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:50.427619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:50.427899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:50.428198 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] PATH_INFO: `/v3/users/8ccea90a4c5c4fdeb7e53af55f102a5b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:50.429357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccea90a4c5c4fdeb7e53af55f102a5b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:50.431291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:50.526245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906e4bc4e9cc45ecb01cfca7918e461d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.533254 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906e4bc4e9cc45ecb01cfca7918e461d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.595392 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccea90a4c5c4fdeb7e53af55f102a5b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.602136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccea90a4c5c4fdeb7e53af55f102a5b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.602870 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccea90a4c5c4fdeb7e53af55f102a5b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.603567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccea90a4c5c4fdeb7e53af55f102a5b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.604244 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906e4bc4e9cc45ecb01cfca7918e461d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.605118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906e4bc4e9cc45ecb01cfca7918e461d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.605972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-556c34cb-7974-4445-9e96-ec4451fb9e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906e4bc4e9cc45ecb01cfca7918e461d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.625016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1252/2510] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:50 2022] DELETE /identity/v3/users/906e4bc4e9cc45ecb01cfca7918e461d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:50.678125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccea90a4c5c4fdeb7e53af55f102a5b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:50.687976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:50.697704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1259/2511] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:50 2022] DELETE /identity/v3/users/8ccea90a4c5c4fdeb7e53af55f102a5b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:50.702199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:50.702494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:50.702809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:50.721338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:50.722585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:50.768407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:50.784470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1253/2512] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:50.794297 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:50.795754 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:50.853361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5854c10-fd79-4a3b-a3ce-992e0fc98127 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:50.858349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:50.858349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:50.858721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:50.858721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:50.858721 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:50.859094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:50.869825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:50.871145 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bcba1-446c-4b1f-b3bc-f7236e8cabb2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:50.871294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a8761-7eee-4db2-8168-e82e4f93c454 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:50.876030 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:50.878261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:50.878261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:50.878261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] PATH_INFO: `/v3/projects/0dc215aadb5345d791945658832aae4a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:50.879486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc215aadb5345d791945658832aae4a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:50.881410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:50.883572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1260/2513] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:51.050835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc215aadb5345d791945658832aae4a', 'request_id': 'req-632aa10f-1c6b-4cb0-86a0-d4071c012bde'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.129198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc215aadb5345d791945658832aae4a', 'request_id': 'req-632aa10f-1c6b-4cb0-86a0-d4071c012bde'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.136160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc215aadb5345d791945658832aae4a', 'request_id': 'req-632aa10f-1c6b-4cb0-86a0-d4071c012bde'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.148689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] The token cache is being invalidate because project 0dc215aadb5345d791945658832aae4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:51.148975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.149676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-632aa10f-1c6b-4cb0-86a0-d4071c012bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.151263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1254/2514] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:50 2022] DELETE /identity/v3/projects/0dc215aadb5345d791945658832aae4a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:51.156227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.156516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.156794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.181716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:51.182961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.246603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1261/2515] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:51.254829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.255123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.255403 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.274337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:51.275564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.284226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1255/2516] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:51.288618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.298620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7270396-da65-44b7-815e-c0c25048b8da None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.303867 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.304544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.304896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.305302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.306021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:53:51.306322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.307941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.314072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.322882 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.323141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1262/2517] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:53:51.326358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.329754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90282d9f-f4fd-4842-8589-4b0458807d89 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.335572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.336910 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.337217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.337756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] PATH_INFO: `/v3/projects/802ab49587934150af9e61c70d2fc06b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.337869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03254-f936-4d4f-a538-9bcdb22e02f5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.339257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802ab49587934150af9e61c70d2fc06b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.341785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.343482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.346742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.346742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.346742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.346742 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.348464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.449819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1263/2518] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:51.453857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.468434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802ab49587934150af9e61c70d2fc06b', 'request_id': 'req-3c709133-5484-4feb-8dbd-4af951f09c24'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.513495 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:51.517056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655db9-e516-46f7-9e71-a48053de9051 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.534327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802ab49587934150af9e61c70d2fc06b', 'request_id': 'req-3c709133-5484-4feb-8dbd-4af951f09c24'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.540743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802ab49587934150af9e61c70d2fc06b', 'request_id': 'req-3c709133-5484-4feb-8dbd-4af951f09c24'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.552673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] The token cache is being invalidate because project 802ab49587934150af9e61c70d2fc06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:51.552962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.553623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.555216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1256/2519] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:51 2022] DELETE /identity/v3/projects/802ab49587934150af9e61c70d2fc06b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:51.559069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.560401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.573941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.574635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.574914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.575183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.575884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.577763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.610413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:51.625202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c709133-5484-4feb-8dbd-4af951f09c24 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.630789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.631659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.631941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.632216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] PATH_INFO: `/v3/users/37f4822be380498e993a34d6464cc216` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.633397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f4822be380498e993a34d6464cc216)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.635369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.650591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1264/2520] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:51.654846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.656437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.668581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d2b0-7b63-43a1-a0a3-714ea0582c45 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.673422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.674379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.674379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.680730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.680730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.680730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.686999 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1265/2521] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:51.691520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.702117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296093c4-f3c1-4367-9ea2-85d84b06b769 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.707406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.715105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.715105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.715105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.715105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.715105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.720982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1266/2522] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:51.725010 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.734684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eb6d0-3bb4-43cd-ac19-c42ae51249e3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:51.739869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:51.740727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.741004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.741272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] PATH_INFO: `/v3/projects/c2ba19e3568c4eefae0b038184f6953e/users/f6d8d77e14ba4ed4a24f1ced7db89eaf/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.756889 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ba19e3568c4eefae0b038184f6953e, user_id=f6d8d77e14ba4ed4a24f1ced7db89eaf, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:51.759555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691600d-c748-44b3-aef7-e9810759f4b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:51.832293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1267/2523] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:51 2022] PUT /identity/v3/projects/c2ba19e3568c4eefae0b038184f6953e/users/f6d8d77e14ba4ed4a24f1ced7db89eaf/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:51.838420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f4822be380498e993a34d6464cc216'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.843231 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:51.843519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:51.843800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:51.844555 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f4822be380498e993a34d6464cc216'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.870785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] MFA Rules not processed for user `f6d8d77e14ba4ed4a24f1ced7db89eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:51.872004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:51.906383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f4822be380498e993a34d6464cc216'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.907067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f4822be380498e993a34d6464cc216'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.907697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f4822be380498e993a34d6464cc216'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:51.927791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1257/2524] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:51 2022] DELETE /identity/v3/users/37f4822be380498e993a34d6464cc216 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:51.936286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1268/2525] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:51.941033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:51.942481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.023269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4317cc-4dd8-419a-b46d-09891ae41b95 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.028116 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.028896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.029186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.029460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.030164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.032148 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.175414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1258/2526] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:51 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:52.176387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.225273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:52.247610 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c2bbf-dd06-4e54-87fe-f627bb78ffed None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.305208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.305877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.306153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.306427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.307154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.309047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.390106 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1269/2527] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:52 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:52.391134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.393703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.396571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.407167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e533d4-625f-4e5b-90c0-2d7c5483a703 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.411090 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20630df-0945-44bb-991d-7f1ce8264a8e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.414798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.414798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.414798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.414798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.415282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.416325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.417136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.417994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.418284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.418581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] PATH_INFO: `/v3/projects/c057f0f966bb47738340dbbc2d60b19a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.419894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c057f0f966bb47738340dbbc2d60b19a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.421783 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.428371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1259/2528] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:52.432611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.443512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782fbc-2bfb-432d-9e11-8c09248c4e84 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.448338 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.449391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.449672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.449947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] PATH_INFO: `/v3/projects/41e1746f270d45ea9086324b091f7751/users/fac7ffc034d2434eb056183fc5f0e1af/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.465303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e1746f270d45ea9086324b091f7751, user_id=fac7ffc034d2434eb056183fc5f0e1af, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.467995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.553779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1260/2529] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:52 2022] PUT /identity/v3/projects/41e1746f270d45ea9086324b091f7751/users/fac7ffc034d2434eb056183fc5f0e1af/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:52.557562 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.563428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c057f0f966bb47738340dbbc2d60b19a', 'request_id': 'req-20680eca-7f66-4ef3-b46a-8010f39b060b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:52.577669 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:52.620144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd348-a151-4a18-b723-c33471735230 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.674875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.675537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.675896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.676178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.676893 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.678478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.688222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1261/2530] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:52.692419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.701849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591278-7b80-46d8-b17e-d2f4a16fbe05 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.706948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.707795 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.708067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.708337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] PATH_INFO: `/v3/projects/41e1746f270d45ea9086324b091f7751/users/fac7ffc034d2434eb056183fc5f0e1af/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.710251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e1746f270d45ea9086324b091f7751, user_id=fac7ffc034d2434eb056183fc5f0e1af, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.713108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17933485-214a-4e28-a0ca-1918987f942f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.724243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1262/2531] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:52 2022] PUT /identity/v3/projects/41e1746f270d45ea9086324b091f7751/users/fac7ffc034d2434eb056183fc5f0e1af/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:52.729503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c057f0f966bb47738340dbbc2d60b19a', 'request_id': 'req-20680eca-7f66-4ef3-b46a-8010f39b060b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:52.734290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.734613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.734916 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.736488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c057f0f966bb47738340dbbc2d60b19a', 'request_id': 'req-20680eca-7f66-4ef3-b46a-8010f39b060b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:52.746897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] The token cache is being invalidate because project c057f0f966bb47738340dbbc2d60b19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:52.747190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:52.747843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-20680eca-7f66-4ef3-b46a-8010f39b060b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:52.749408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1270/2532] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:52 2022] DELETE /identity/v3/projects/c057f0f966bb47738340dbbc2d60b19a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:52.759458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028425-7cea-4d4c-95f4-46721b42c8d2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.759887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028425-7cea-4d4c-95f4-46721b42c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.760173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028425-7cea-4d4c-95f4-46721b42c8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.762785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] MFA Rules not processed for user `fac7ffc034d2434eb056183fc5f0e1af`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:52.764033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.778226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-13028425-7cea-4d4c-95f4-46721b42c8d2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:52.779439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13028425-7cea-4d4c-95f4-46721b42c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.831839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:52.833592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1263/2533] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:52.841666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1271/2534] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:52.845604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.847831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.889911 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:52.909145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.909475 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.909740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.914315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c43b1-ab5c-4a72-92e4-cf9b41e1ff09 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.918028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.918692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.918979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.919256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.927692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:52.928963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:52.930074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.932245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.937063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1272/2535] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:52.941900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.942938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1264/2536] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:53:52.951298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78996349-4aca-4aa1-8c82-8e1ec2badb90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.956796 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.957840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.958566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.959102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.959855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:53:52.960168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.961860 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:52.974903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1273/2537] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:53:52.979329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:52.990207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31b096-f6da-4597-bfbb-c88c5ac4937e service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:52.994980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:52.995619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:52.995902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:52.996175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:52.996899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:52.998838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.102644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1265/2538] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:52 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:53.106569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.117014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a11f25-1627-4e35-a06e-0c0e9965d976 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.136200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.177426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.178130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.178416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.178729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.179412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.181325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.195731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432166b3-6e53-4958-b716-79672b9aa450 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.199567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.200218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.200502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.200781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.202573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:53.249606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1274/2539] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:53 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:53.255260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.266627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb1910-39ad-4346-9d0f-2cae943c078f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.272930 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.273635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.273924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.274203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.276668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.276800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.276891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.278580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.287120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1275/2540] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:53.291287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.295693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1266/2541] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:53:53.301273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea960c2-6689-45eb-84ca-1b940e9760a1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.306509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.307444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.307899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.308349 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.309428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.311343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f853cb-ca38-474b-b565-3e38bda0f596 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.323350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1276/2542] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:53.327776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.343731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24f9e6-633c-4670-8454-003d3610bbc4 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.351170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.352199 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.352541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.352881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] PATH_INFO: `/v3/projects/c5945b9650e0443f9bd7ef2be7306a6f/users/e7599d72d03f4e749e389d18373b565d/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.368780 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5945b9650e0443f9bd7ef2be7306a6f, user_id=e7599d72d03f4e749e389d18373b565d, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.372668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.454230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1267/2543] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:53 2022] PUT /identity/v3/projects/c5945b9650e0443f9bd7ef2be7306a6f/users/e7599d72d03f4e749e389d18373b565d/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:53.464814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.465103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.465376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.494479 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] MFA Rules not processed for user `e7599d72d03f4e749e389d18373b565d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:53.495725 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:53.565294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1277/2544] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:53.570110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.619042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:53.636537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b3ba-5efc-4de9-a238-8f5190ea66c2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.641180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.641833 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.642115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.642415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.643212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.645118 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.743334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1268/2545] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:53 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:53.747337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.758637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08114-7124-4d33-9cde-8344bb23914a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.790608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:53.815282 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.815982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.816369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.816648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.817342 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.819352 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.913377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1278/2546] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:53 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:53.917543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.928765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc36ec-0924-4df7-96da-2ce365fa6bd1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.933438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.934074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.934352 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.934659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.935379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.936978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.948647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1269/2547] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:53.952846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.963438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac099-38f4-4c3f-a2af-04eef7b0198a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:53.967987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:53.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:53.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:53.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:53.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:53.972439 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6df00-d34a-4d9b-90cf-b7aecfa9b60f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:53.982501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1279/2548] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:53.987760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:53.997931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615bfa8-0e5d-47cd-8253-8f80ec05de6e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.002827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.003705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.004033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.004264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] PATH_INFO: `/v3/projects/ceadef4400c2460aa156650cf07e0e4c/users/770b55fb4159454091215257fe507de5/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.020367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceadef4400c2460aa156650cf07e0e4c, user_id=770b55fb4159454091215257fe507de5, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.023028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.112593 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1270/2549] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:53 2022] PUT /identity/v3/projects/ceadef4400c2460aa156650cf07e0e4c/users/770b55fb4159454091215257fe507de5/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:54.124089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.124382 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.124670 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.150273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] MFA Rules not processed for user `770b55fb4159454091215257fe507de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:54.151507 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:54.216239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1280/2550] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:54.221580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.281902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53c00-6384-4c2d-aea3-8ba7b673286e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.287374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.288041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.288325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.288602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.289304 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.291258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.417549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1271/2551] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:54 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:54.421544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.432401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f0f2b-4fe9-438f-a577-e303c59e0773 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.487269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.488752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.488752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.488752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.489298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.491238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.551800 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1281/2552] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:54 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:54.555874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.567532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef2eeb-0570-4637-a626-5e8c887064b5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.572252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.572907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.573190 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.573465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.574178 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.575848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.587750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1272/2553] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:54.591894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.602124 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf4477-8270-4184-b5ae-b4cc549bfa33 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.606681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.607548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.607832 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.608109 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] PATH_INFO: `/v3/projects/57b0b27fe45b48259f795554b8d2ddaf/users/b545dc73fc294667a72c6dd47b1c687e/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.622254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b0b27fe45b48259f795554b8d2ddaf, user_id=b545dc73fc294667a72c6dd47b1c687e, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.624926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.701084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1282/2554] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:54 2022] PUT /identity/v3/projects/57b0b27fe45b48259f795554b8d2ddaf/users/b545dc73fc294667a72c6dd47b1c687e/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:54.704589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.715458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525d85b-f94d-484e-af0d-1baf1f2e3da1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.774477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.775225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.775508 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.775782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.776972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.778127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd00fc-d655-4b52-a907-d3fba8ee349a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.789163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1273/2555] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:54.793431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:54.804137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d415d3-86d3-4d3b-a9f2-36491d2e21da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:54.808807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:54.809679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.809959 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.810239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] PATH_INFO: `/v3/projects/57b0b27fe45b48259f795554b8d2ddaf/users/b545dc73fc294667a72c6dd47b1c687e/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.812277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b0b27fe45b48259f795554b8d2ddaf, user_id=b545dc73fc294667a72c6dd47b1c687e, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:54.815546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:54.818654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:54.826189 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1283/2556] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:54 2022] PUT /identity/v3/projects/57b0b27fe45b48259f795554b8d2ddaf/users/b545dc73fc294667a72c6dd47b1c687e/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:54.837240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:54.837792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:54.838068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:54.845017 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:54.864140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] MFA Rules not processed for user `b545dc73fc294667a72c6dd47b1c687e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:54.865374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:54.935828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1274/2557] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:54.947848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:55.003603 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759aee-a686-478b-a4f3-52ed2f1e601e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:55.007633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:55.008526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:55.008874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:55.009724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:55.024665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:55.024665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:55.032166 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1284/2558] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:55.171743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:55.182361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef16f0-2e98-48bc-8580-2f1b3d2e9ca9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:55.186082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:55.186751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:55.187034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:55.187313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:55.246924 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:55.248397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b79b0-adb9-485b-9434-a900648103e6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:55.261169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1275/2559] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:56.865192 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:56.866673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:56.932531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9c403-2150-4523-b45a-795a04aecfeb service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:56.937058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:56.937920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:56.938416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:56.938727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] PATH_INFO: `/v3/users/f6d8d77e14ba4ed4a24f1ced7db89eaf` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:56.940560 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d8d77e14ba4ed4a24f1ced7db89eaf)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:56.942565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:57.132529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d8d77e14ba4ed4a24f1ced7db89eaf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:57.139601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d8d77e14ba4ed4a24f1ced7db89eaf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:57.140268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d8d77e14ba4ed4a24f1ced7db89eaf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:57.140893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d8d77e14ba4ed4a24f1ced7db89eaf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:57.192008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d8d77e14ba4ed4a24f1ced7db89eaf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:57.210346 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1285/2560] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:56 2022] DELETE /identity/v3/users/f6d8d77e14ba4ed4a24f1ced7db89eaf => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:57.215251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:57.215739 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:57.215841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:57.234708 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:57.240836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:57.297282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1276/2561] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:57.313233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:57.314666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:57.338895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:57.378065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb5eb-4f2b-45ec-9501-150f215b1368 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:57.381697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:57.382338 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:57.382650 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:57.382929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:57.392267 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:57.396445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:57.403820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1286/2562] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:53:57.799675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:57.800913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:57.813316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f054a-1f51-4567-835e-c77d20eadc03 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:57.819079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:57.820053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:57.820243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:57.820519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] PATH_INFO: `/v3/projects/c2ba19e3568c4eefae0b038184f6953e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:57.822069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ba19e3568c4eefae0b038184f6953e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:57.823810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:57.852195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:57.969516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ba19e3568c4eefae0b038184f6953e', 'request_id': 'req-566e0db4-9995-4613-8b69-c0ccfc84401e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.054858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ba19e3568c4eefae0b038184f6953e', 'request_id': 'req-566e0db4-9995-4613-8b69-c0ccfc84401e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.060858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ba19e3568c4eefae0b038184f6953e', 'request_id': 'req-566e0db4-9995-4613-8b69-c0ccfc84401e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.072291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] The token cache is being invalidate because project c2ba19e3568c4eefae0b038184f6953e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:58.072580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.073208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.074962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1277/2563] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:57 2022] DELETE /identity/v3/projects/c2ba19e3568c4eefae0b038184f6953e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:58.078993 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:58.091325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e432ea-2c06-4058-8f92-f87ca0574fa5 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:58.149086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:58.149958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:58.150239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:58.150683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] PATH_INFO: `/v3/users/fac7ffc034d2434eb056183fc5f0e1af` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:58.151797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac7ffc034d2434eb056183fc5f0e1af)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:58.153691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:58.279523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac7ffc034d2434eb056183fc5f0e1af'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.287324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac7ffc034d2434eb056183fc5f0e1af'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.288051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac7ffc034d2434eb056183fc5f0e1af'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.288661 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac7ffc034d2434eb056183fc5f0e1af'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.329070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0c7b6173-61f6-4b85-98c9-1d880aff08df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac7ffc034d2434eb056183fc5f0e1af'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.347735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1287/2564] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:58 2022] DELETE /identity/v3/users/fac7ffc034d2434eb056183fc5f0e1af => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:58.730931 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:58.732487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:58.746395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e0db4-9995-4613-8b69-c0ccfc84401e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:58.811297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:58.812074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:58.812359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:58.812634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] PATH_INFO: `/v3/projects/41e1746f270d45ea9086324b091f7751` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:58.813800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e1746f270d45ea9086324b091f7751)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:58.816442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:58.927582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e1746f270d45ea9086324b091f7751', 'request_id': 'req-87235673-a451-41dc-85b3-9bde0d317ed5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.991707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e1746f270d45ea9086324b091f7751', 'request_id': 'req-87235673-a451-41dc-85b3-9bde0d317ed5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:58.997724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e1746f270d45ea9086324b091f7751', 'request_id': 'req-87235673-a451-41dc-85b3-9bde0d317ed5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.009850 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] The token cache is being invalidate because project 41e1746f270d45ea9086324b091f7751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:53:59.010138 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.010809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87235673-a451-41dc-85b3-9bde0d317ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.012347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1278/2565] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:53:58 2022] DELETE /identity/v3/projects/41e1746f270d45ea9086324b091f7751 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:53:59.017222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.017513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.018344 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.036760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:59.037934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.071532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:59.097143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1288/2566] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:59.106387 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.106634 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.106943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.125309 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:59.126560 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.135364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1279/2567] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:53:59.139519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.150001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d40dc-0620-4cba-8cf4-ea5e5e738f90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.154639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.155291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.155574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.155852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.156741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:53:59.157312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.159899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.173746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1289/2568] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:53:59.178307 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.188065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7eeeb1-2ae7-43d2-8cdf-ad97f1c7ca27 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.193597 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.194340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.194727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.195007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.195769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.197896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.287679 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.289447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.296854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1280/2569] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:53:59.302146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83890bf-a06f-4685-a457-2345f5b3b29a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.305495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.319584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:59.361563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.362486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.362819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.363096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] PATH_INFO: `/v3/users/e7599d72d03f4e749e389d18373b565d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.364313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7599d72d03f4e749e389d18373b565d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.366199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.370875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63f55f-2df6-4b2d-b72e-54384af7ab8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.376120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.376782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.377577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.378183 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.378572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.380459 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.455031 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1281/2570] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:59.459424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.469265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8a86-6a79-425a-ab6f-1f6981cd61e5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.474667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.475314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.475596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.475869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.476582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.478166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.488713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1282/2571] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:53:59.492569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.503465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a4ed4-bd39-4c4d-b886-b55016317488 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.507536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.508152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.508816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.508816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.509431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.511050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.522150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1283/2572] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:53:59.526051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.536248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6eb1b-99fb-46f8-87fe-55660b59a44b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.540950 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.541808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.541921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7599d72d03f4e749e389d18373b565d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.542089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.542365 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] PATH_INFO: `/v3/projects/62f308aa1d12468e9a71c7441c0a3718/users/ab4767d405384b9eb0b9c8099d2ef9a4/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.548741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7599d72d03f4e749e389d18373b565d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.549640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7599d72d03f4e749e389d18373b565d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.550266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7599d72d03f4e749e389d18373b565d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.557239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f308aa1d12468e9a71c7441c0a3718, user_id=ab4767d405384b9eb0b9c8099d2ef9a4, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.559918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3552639-4b49-4d42-af0e-df6e14bd72d3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.614972 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d5d4717-45fe-4e9a-8529-b5293079d4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7599d72d03f4e749e389d18373b565d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:53:59.632563 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1290/2573] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:53:59 2022] DELETE /identity/v3/users/e7599d72d03f4e749e389d18373b565d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:59.637174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.637461 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.637738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.644649 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1284/2574] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:53:59 2022] PUT /identity/v3/projects/62f308aa1d12468e9a71c7441c0a3718/users/ab4767d405384b9eb0b9c8099d2ef9a4/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:53:59.656093 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:59.657631 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.659297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.659632 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.659946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.687792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] MFA Rules not processed for user `ab4767d405384b9eb0b9c8099d2ef9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:53:59.689045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.699815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:53:59.717968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1291/2575] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:59.728195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.729561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.759410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1285/2576] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:53:59.763874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.765171 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:53:59.779182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339ea44-1a0e-48ae-a431-c555d0f23287 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.784558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.785297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.785580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.786525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.786525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.788510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.789829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab31fa-6305-4cc7-ba70-0cc9fa11c933 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.793502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.794142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.794435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.794741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.805519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.812364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:53:59.816984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1292/2577] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:53:59.897762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1286/2578] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:53:59.901868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:53:59.912328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251dd8-5c4a-4161-974a-5230e2c1b52a service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:53:59.967200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:53:59.967864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:53:59.968142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:53:59.968414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:53:59.969103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:53:59.971646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.082179 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1293/2579] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:53:59 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:00.083265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.094714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001692-8af2-4093-a18b-f133ea33eab4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.100040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.100699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.100980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.101253 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.101981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.104181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.116247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1287/2580] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:00.120590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.131284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f0598-49f0-409d-b8ec-71e06dcd6f30 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.136158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.137035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.137373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.137634 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] PATH_INFO: `/v3/projects/427a7f460ba745feb8b3cb8c37496c86/users/1b4eaa38cf554fd3a56956fe922fa049/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.152920 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427a7f460ba745feb8b3cb8c37496c86, user_id=1b4eaa38cf554fd3a56956fe922fa049, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.155553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.217328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.219191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:00.233296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8261e-42e1-4501-b326-a45b83978085 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.244718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:00.253582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1294/2581] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:00 2022] PUT /identity/v3/projects/427a7f460ba745feb8b3cb8c37496c86/users/1b4eaa38cf554fd3a56956fe922fa049/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:00.254639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.296502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.297357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.297635 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.297902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] PATH_INFO: `/v3/projects/c5945b9650e0443f9bd7ef2be7306a6f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.299769 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5945b9650e0443f9bd7ef2be7306a6f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.301667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.315073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:00.320168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e98c-cc0a-415e-b2ea-cd96078442c8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.324708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.325388 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.325666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.325960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.326691 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.328308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.338940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1295/2582] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:00.343190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.353313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb06-214d-4262-ac96-65dfc67a7fda admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.358122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.359056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.359335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.359606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] PATH_INFO: `/v3/projects/427a7f460ba745feb8b3cb8c37496c86/users/1b4eaa38cf554fd3a56956fe922fa049/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.361803 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427a7f460ba745feb8b3cb8c37496c86, user_id=1b4eaa38cf554fd3a56956fe922fa049, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.364665 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e452a9-280b-45b2-9f24-5386450ec3f3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.374874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1296/2583] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:00 2022] PUT /identity/v3/projects/427a7f460ba745feb8b3cb8c37496c86/users/1b4eaa38cf554fd3a56956fe922fa049/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:00.385233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.385518 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.385797 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.400955 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5945b9650e0443f9bd7ef2be7306a6f', 'request_id': 'req-3e7d0192-fad9-4841-8c48-61f32daee4a7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.413984 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] MFA Rules not processed for user `1b4eaa38cf554fd3a56956fe922fa049`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:00.415268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:00.470681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5945b9650e0443f9bd7ef2be7306a6f', 'request_id': 'req-3e7d0192-fad9-4841-8c48-61f32daee4a7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.478619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5945b9650e0443f9bd7ef2be7306a6f', 'request_id': 'req-3e7d0192-fad9-4841-8c48-61f32daee4a7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.483545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1297/2584] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:00 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:00.490434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] The token cache is being invalidate because project c5945b9650e0443f9bd7ef2be7306a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:00.490830 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.491493 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.493595 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1288/2585] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:00 2022] DELETE /identity/v3/projects/c5945b9650e0443f9bd7ef2be7306a6f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:00.494400 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.495732 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:00.496968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:00.498249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:00.552444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafd0c9-851f-4cb9-a250-aab1118b8d55 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.556211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.556851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.560403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.560403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.560403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:00.569819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d0192-fad9-4841-8c48-61f32daee4a7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:00.572624 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.574058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.574486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:00.575424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:00.575693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:00.575967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] PATH_INFO: `/v3/users/770b55fb4159454091215257fe507de5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:00.577151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770b55fb4159454091215257fe507de5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:00.579092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:00.584926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1298/2586] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:00 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:54:00.738487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:00.740268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770b55fb4159454091215257fe507de5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.747174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770b55fb4159454091215257fe507de5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.825424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770b55fb4159454091215257fe507de5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.826191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770b55fb4159454091215257fe507de5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.826765 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770b55fb4159454091215257fe507de5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:00.847102 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1289/2587] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:00 2022] DELETE /identity/v3/users/770b55fb4159454091215257fe507de5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:01.324678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:01.338395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead2450-fd1f-491c-9002-f1edad3f64da service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:01.405309 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:01.406193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:01.406459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:01.406766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] PATH_INFO: `/v3/projects/ceadef4400c2460aa156650cf07e0e4c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:01.408064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceadef4400c2460aa156650cf07e0e4c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:01.409978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:01.417974 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:01.512987 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceadef4400c2460aa156650cf07e0e4c', 'request_id': 'req-27a7be50-de6f-4180-b997-853193aaa525'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.574513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceadef4400c2460aa156650cf07e0e4c', 'request_id': 'req-27a7be50-de6f-4180-b997-853193aaa525'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.582039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceadef4400c2460aa156650cf07e0e4c', 'request_id': 'req-27a7be50-de6f-4180-b997-853193aaa525'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.592424 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] The token cache is being invalidate because project ceadef4400c2460aa156650cf07e0e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:01.592711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.593328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.594861 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1299/2588] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:01 2022] DELETE /identity/v3/projects/ceadef4400c2460aa156650cf07e0e4c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:01.599206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:01.600590 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:01.614534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd23ef-7b9d-4e41-a92b-ab57081af774 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:01.672569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:01.673445 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:01.673754 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:01.674781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] PATH_INFO: `/v3/users/b545dc73fc294667a72c6dd47b1c687e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:01.675988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b545dc73fc294667a72c6dd47b1c687e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:01.678020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:01.875126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b545dc73fc294667a72c6dd47b1c687e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.882963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b545dc73fc294667a72c6dd47b1c687e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.963415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b545dc73fc294667a72c6dd47b1c687e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.964166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b545dc73fc294667a72c6dd47b1c687e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.964836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-910067ba-945e-47e1-a8db-1c4fc5e32a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b545dc73fc294667a72c6dd47b1c687e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:01.983209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1290/2589] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:01 2022] DELETE /identity/v3/users/b545dc73fc294667a72c6dd47b1c687e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:02.427967 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:02.433692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:02.442451 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7be50-de6f-4180-b997-853193aaa525 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:02.498085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:02.498977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:02.499254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:02.499523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] PATH_INFO: `/v3/projects/57b0b27fe45b48259f795554b8d2ddaf` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:02.500788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b0b27fe45b48259f795554b8d2ddaf)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:02.502809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:02.638983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b0b27fe45b48259f795554b8d2ddaf', 'request_id': 'req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:02.746105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b0b27fe45b48259f795554b8d2ddaf', 'request_id': 'req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:02.757651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b0b27fe45b48259f795554b8d2ddaf', 'request_id': 'req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:02.764768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] The token cache is being invalidate because project 57b0b27fe45b48259f795554b8d2ddaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:02.765047 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:02.765689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54e9b0a5-32f2-46d1-b693-8ebd9d3d8e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:02.767266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1300/2590] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:02 2022] DELETE /identity/v3/projects/57b0b27fe45b48259f795554b8d2ddaf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:02.774055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:02.774340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:02.774656 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:02.794886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:02.796078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:02.835196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:02.858323 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1291/2591] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:02.867042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:02.867343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:02.867626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:02.886752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:02.888014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:02.896824 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1301/2592] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:02.901154 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:02.911172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d8804-ae7c-46b5-9068-bdf26cc80815 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:02.915724 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:02.916367 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:02.916645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:02.916918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:02.917687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:02.917990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:02.919604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:02.931996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1292/2593] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:54:02.936502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:02.946602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e3a3-aeb8-4df6-8243-053c4aaec412 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:02.951266 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:02.951906 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:02.952182 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:02.952453 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:02.953142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:02.955141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.077907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:03.089438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1302/2594] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:02 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:03.090854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.102205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15374c-6156-4b5a-9da4-e89dc6931aa7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.167149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.167817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.168432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.168432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.169034 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.170984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.255981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1293/2595] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:03 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:03.260274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.270786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf4be3-e1a4-45fe-8549-a70fb82d98fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.275460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.276144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.276425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.276701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.277473 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.279078 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.289809 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1303/2596] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:03.294098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.305091 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1f03-689c-4e43-be5c-06722b846370 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.309631 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.310277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.310582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.310858 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.311601 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.313167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99706d1e-70ae-4382-8571-6dcf49819407 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.324143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1294/2597] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:03.328430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.339104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07facfa4-4bf1-49b7-9277-68dbf15de3e6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.343858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.344604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.344888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.345274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] PATH_INFO: `/v3/projects/0f5f72a0f2054b89bb61968a2de9f10a/users/1d04261c643948c0b7d9a20579c52dae/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.360551 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5f72a0f2054b89bb61968a2de9f10a, user_id=1d04261c643948c0b7d9a20579c52dae, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.364668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.460582 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1304/2598] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:03 2022] PUT /identity/v3/projects/0f5f72a0f2054b89bb61968a2de9f10a/users/1d04261c643948c0b7d9a20579c52dae/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:03.465971 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.466260 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.466537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.484491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:03.485668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:03.547716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1295/2599] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:03.558810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.560513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:03.624086 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f699d-9b10-4c22-b518-efb6cd9967a6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.625197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.625863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.626107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.626383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.636979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.641790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2cb07-f14d-4681-88b0-272dc078e6c9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.648221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1305/2600] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:03.767752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:03.769556 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:03.782115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7a12-63ec-4986-9d46-8383072d5f7a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:03.787650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:03.788520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:03.788798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:03.793438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] PATH_INFO: `/v3/users/ab4767d405384b9eb0b9c8099d2ef9a4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:03.793438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4767d405384b9eb0b9c8099d2ef9a4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:03.793438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:03.981696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:03.984414 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4767d405384b9eb0b9c8099d2ef9a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:03.991719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4767d405384b9eb0b9c8099d2ef9a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:04.064349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4767d405384b9eb0b9c8099d2ef9a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:04.064974 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4767d405384b9eb0b9c8099d2ef9a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:04.065584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4767d405384b9eb0b9c8099d2ef9a4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:04.083377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1296/2601] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:03 2022] DELETE /identity/v3/users/ab4767d405384b9eb0b9c8099d2ef9a4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:04.088385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:04.088680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:04.088991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:04.108084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:04.109571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:04.173884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1306/2602] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:04.186285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:04.191847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:04.255340 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e2e00-a578-4c20-af05-18a0689b4a0d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:04.259075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:04.259726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:04.260012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:04.260291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:04.269717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:04.271487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe4c02-fc4e-4b40-934c-c3a516ba60f0 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:04.281107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1297/2603] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:04.886478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:04.888212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:04.901277 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ea110-c044-4fdb-b293-4943f61345b1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:04.906058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:04.907163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:04.907519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:04.912262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] PATH_INFO: `/v3/projects/62f308aa1d12468e9a71c7441c0a3718` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:04.912262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f308aa1d12468e9a71c7441c0a3718)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:04.912262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:05.035445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f308aa1d12468e9a71c7441c0a3718', 'request_id': 'req-d3adcc61-d5e9-49ba-973b-d35ba712591a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.057538 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:05.058534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:05.058889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:05.090197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] MFA Rules not processed for user `1d04261c643948c0b7d9a20579c52dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:05.091455 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:05.100173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f308aa1d12468e9a71c7441c0a3718', 'request_id': 'req-d3adcc61-d5e9-49ba-973b-d35ba712591a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.108387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f308aa1d12468e9a71c7441c0a3718', 'request_id': 'req-d3adcc61-d5e9-49ba-973b-d35ba712591a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.109786 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:05.120434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] The token cache is being invalidate because project 62f308aa1d12468e9a71c7441c0a3718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:05.120726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.121778 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.123126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1307/2604] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:04 2022] DELETE /identity/v3/projects/62f308aa1d12468e9a71c7441c0a3718 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:05.127077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:05.129682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:05.171408 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1298/2605] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:05.183127 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:05.184549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:05.205134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adcc61-d5e9-49ba-973b-d35ba712591a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:05.210130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:05.211033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:05.211313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:05.211587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] PATH_INFO: `/v3/users/1b4eaa38cf554fd3a56956fe922fa049` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:05.213138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4eaa38cf554fd3a56956fe922fa049)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:05.215064 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:05.247812 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17258d-9e3c-45fa-94f0-c164450283e8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:05.251512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:05.252156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:05.252439 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:05.252715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:05.259745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:05.262036 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:05.263528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:05.273603 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1299/2606] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:54:05.366164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4eaa38cf554fd3a56956fe922fa049'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.376491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4eaa38cf554fd3a56956fe922fa049'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.377130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4eaa38cf554fd3a56956fe922fa049'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.377749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4eaa38cf554fd3a56956fe922fa049'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.465894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77c84fe4-7405-43c2-9ca8-ac29f57b7e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4eaa38cf554fd3a56956fe922fa049'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:05.484828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1308/2607] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:05 2022] DELETE /identity/v3/users/1b4eaa38cf554fd3a56956fe922fa049 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:05.931654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:05.946149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81183e-8e22-4935-892e-433bdd680181 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.012740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.013602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.013878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.014149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] PATH_INFO: `/v3/projects/427a7f460ba745feb8b3cb8c37496c86` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.015347 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=427a7f460ba745feb8b3cb8c37496c86)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.017258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.152618 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427a7f460ba745feb8b3cb8c37496c86', 'request_id': 'req-3a040888-f606-4d95-9c77-4f15ba9f9cbe'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:06.255676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427a7f460ba745feb8b3cb8c37496c86', 'request_id': 'req-3a040888-f606-4d95-9c77-4f15ba9f9cbe'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:06.261999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427a7f460ba745feb8b3cb8c37496c86', 'request_id': 'req-3a040888-f606-4d95-9c77-4f15ba9f9cbe'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:06.273998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] The token cache is being invalidate because project 427a7f460ba745feb8b3cb8c37496c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:06.274282 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:06.274918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3a040888-f606-4d95-9c77-4f15ba9f9cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:06.276506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1300/2608] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:05 2022] DELETE /identity/v3/projects/427a7f460ba745feb8b3cb8c37496c86 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:06.281421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.281884 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.281980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.301534 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:06.302914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:06.359911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1309/2609] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:06.368099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.368389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.368666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.388398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:06.389584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:06.397819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1301/2610] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:06.401905 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.411460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0ca75-dae3-4638-bf35-13ef30bd6d66 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.416112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.416760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.417076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.417364 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.418097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:06.418403 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.420033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.422869 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:06.432814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1310/2611] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:54:06.437203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.447168 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2420aef-f22e-455e-86f1-2d8ee56f4593 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.451760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.452404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.452684 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.452963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.453642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.455557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.564719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1302/2612] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:06.569546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.579488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff3db-fa19-4b8e-874f-e27681d15bc0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.637820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.638480 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.638794 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.639190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.639882 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.641888 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.774589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1311/2613] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:06.779169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.789392 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664c350-c25b-48c9-9b16-c581d10b78ee admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.793965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.794781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.795142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.795545 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.796237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.797826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.808732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1303/2614] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:06.812649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.827883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d12b0-0685-498e-9240-174a49dedc75 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.827883 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.828557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.828840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.829142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.829879 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.832128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22d8e7-432b-473f-bee4-08ecd1f6b228 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.842390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1312/2615] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:06.846473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:06.856463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242b13-056f-4fda-ae1b-12731568aca5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:06.861047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:06.861912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.863139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.863139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] PATH_INFO: `/v3/projects/02fc19e9b7a5460bbe77ca73aafc6bc3/users/4cc07031e47441c0a960e590947efa94/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.878592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fc19e9b7a5460bbe77ca73aafc6bc3, user_id=4cc07031e47441c0a960e590947efa94, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:06.881377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e652b-c99e-456c-9bf1-0620ba905ab6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:06.919470 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.919793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.920076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.938695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] MFA Rules not processed for user `1d04261c643948c0b7d9a20579c52dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:06.939875 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:06.951289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1304/2616] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:06 2022] PUT /identity/v3/projects/02fc19e9b7a5460bbe77ca73aafc6bc3/users/4cc07031e47441c0a960e590947efa94/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:06.962611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:06.962737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:06.962938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:06.990796 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] MFA Rules not processed for user `4cc07031e47441c0a960e590947efa94`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:06.992033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:06.999049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1313/2617] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:07.001716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:07.058184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1305/2618] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:07.062521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.074069 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79384f80-591b-4bcc-88e1-6f57ea4eb0a9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.134224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.134980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.135262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.135561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.136255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.138130 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.216941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1314/2619] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:07 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:07.221115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.233212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a60bc-98f7-4120-bcbf-c38729753859 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.291825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.292497 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.292787 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.293055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.293734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.295710 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.376921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1306/2620] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:07 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:07.378820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.385837 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:07.389316 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041cc3b-a559-4180-aa88-c001dde00ad5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.394687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.395415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.395697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.395966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.396676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.398284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.408876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1315/2621] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:07.413427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.423734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef871dca-34f1-40af-8f33-eae56e5140a4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.436491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.436491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.436491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.436491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] PATH_INFO: `/v3/projects/36acb5f2d7074fc899e3be79d3573228/users/f2a740a822964a589baf75aee613825b/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.444888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=36acb5f2d7074fc899e3be79d3573228, user_id=f2a740a822964a589baf75aee613825b, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.447623 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.506834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1307/2622] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:07 2022] PUT /identity/v3/projects/36acb5f2d7074fc899e3be79d3573228/users/f2a740a822964a589baf75aee613825b/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:07.510933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.521673 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fc63d-b1e3-44b0-9a60-0ab30e9bd45b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.583944 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.584735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.585040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.585319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.586044 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.587973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80c544-8032-4257-bd65-d3a98a3df029 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.610835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1316/2623] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:07.611424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.619129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f2382-c080-462d-81b6-5589fb4e4e7c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.624063 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.624938 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.625406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.625581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] PATH_INFO: `/v3/projects/36acb5f2d7074fc899e3be79d3573228/users/f2a740a822964a589baf75aee613825b/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.629980 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36acb5f2d7074fc899e3be79d3573228, user_id=f2a740a822964a589baf75aee613825b, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.632715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.644985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1308/2624] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:07 2022] PUT /identity/v3/projects/36acb5f2d7074fc899e3be79d3573228/users/f2a740a822964a589baf75aee613825b/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:07.667298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.667298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.667298 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.696025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] MFA Rules not processed for user `f2a740a822964a589baf75aee613825b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:07.697281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:07.777032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1317/2625] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:07.788864 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:07.790761 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:07.849236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18b9b-64a8-4855-b510-9fd84a6430e3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:07.852908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:07.853584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:07.853867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:07.854146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:07.864226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:07.866122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0c40d-c4e7-455d-b7c0-38093516db1e service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:07.875108 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1309/2626] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:07.941068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:08.004061 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f78d-e716-43ce-aa58-8e5ad67dcb29 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:08.008724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:08.009649 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:08.009919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:08.010191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] PATH_INFO: `/v3/users/4cc07031e47441c0a960e590947efa94` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:08.011423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc07031e47441c0a960e590947efa94)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:08.013394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:08.030076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:08.211413 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc07031e47441c0a960e590947efa94'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:08.218385 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc07031e47441c0a960e590947efa94'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:08.219054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc07031e47441c0a960e590947efa94'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:08.219669 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc07031e47441c0a960e590947efa94'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:08.294306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc07031e47441c0a960e590947efa94'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:08.312842 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1318/2627] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:07 2022] DELETE /identity/v3/users/4cc07031e47441c0a960e590947efa94 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:08.317846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:08.318370 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:08.318516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:08.330075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:08.336249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:08.337734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:08.396937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1310/2628] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:08.406695 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:08.408141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:08.467261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c15c7-90d0-45ec-a6f9-dd5e8fdfb830 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:08.470890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:08.472326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:08.472620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:08.472904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:08.482262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:08.483733 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:08.493458 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1319/2629] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:08.930693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:08.932146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:09.013400 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1eacd-6481-4a39-8e63-d21ba5740e38 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:09.018067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:09.019879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:09.024075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:09.024075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] PATH_INFO: `/v3/projects/02fc19e9b7a5460bbe77ca73aafc6bc3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:09.024075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fc19e9b7a5460bbe77ca73aafc6bc3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:09.024075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:09.141277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fc19e9b7a5460bbe77ca73aafc6bc3', 'request_id': 'req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.214524 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fc19e9b7a5460bbe77ca73aafc6bc3', 'request_id': 'req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.220942 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fc19e9b7a5460bbe77ca73aafc6bc3', 'request_id': 'req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.232512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] The token cache is being invalidate because project 02fc19e9b7a5460bbe77ca73aafc6bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:09.232809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.233461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.235136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1311/2630] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:08 2022] DELETE /identity/v3/projects/02fc19e9b7a5460bbe77ca73aafc6bc3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:09.240100 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:09.283830 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650da0a-774b-4cc5-ab97-1c78d0c09477 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:09.340306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:09.341181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:09.341464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:09.341743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] PATH_INFO: `/v3/users/f2a740a822964a589baf75aee613825b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:09.342961 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a740a822964a589baf75aee613825b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:09.344865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:09.349323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:09.549043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a740a822964a589baf75aee613825b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.556820 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a740a822964a589baf75aee613825b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.557483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a740a822964a589baf75aee613825b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.558129 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a740a822964a589baf75aee613825b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.608950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f3da6b7f-508f-48ad-9aff-e386f738f5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a740a822964a589baf75aee613825b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:09.631447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1320/2631] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:09 2022] DELETE /identity/v3/users/f2a740a822964a589baf75aee613825b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:10.136092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:10.137531 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:10.151577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1f7c-2ee5-4f42-aec5-8ea92d7b2db4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:10.180570 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:10.211831 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:10.212958 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.213239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.213516 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] PATH_INFO: `/v3/projects/36acb5f2d7074fc899e3be79d3573228` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.214732 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36acb5f2d7074fc899e3be79d3573228)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:10.216696 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:10.352694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36acb5f2d7074fc899e3be79d3573228', 'request_id': 'req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:10.454702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36acb5f2d7074fc899e3be79d3573228', 'request_id': 'req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:10.461577 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36acb5f2d7074fc899e3be79d3573228', 'request_id': 'req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:10.473191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] The token cache is being invalidate because project 36acb5f2d7074fc899e3be79d3573228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:10.473482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:10.474132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0a3899c-bab3-4f95-8db6-c5aa7b67d045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:10.475900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1312/2632] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:10 2022] DELETE /identity/v3/projects/36acb5f2d7074fc899e3be79d3573228 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:10.480806 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.481141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.481422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.539378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:10.540586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:10.605685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1321/2633] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:10.613548 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.613840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.614121 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.634001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:10.635255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:10.643363 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1313/2634] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:10.647681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:10.660221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca48689-1046-4ba8-bf0c-6a767000e5d3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:10.665710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:10.666394 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.666730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.667014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.668287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:10.668287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:10.670087 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:10.686857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1322/2635] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:54:10.691096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:10.706391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eabcd3-f9bc-482d-9512-dfe7a14a82ed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:10.712463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:10.713132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.713417 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.717012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.717012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:10.717012 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:10.814115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1314/2636] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:10 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:10.815509 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:10.826597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf524-a1c1-4503-9630-e724f37e6c47 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:10.875357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:10.896318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:10.896759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:10.897048 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:10.897328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:10.898187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:10.900224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:11.016640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1323/2637] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:10 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:11.020960 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:11.036871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc37ea-42d4-4217-82bc-966b88ee2c3c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:11.042473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:11.043129 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:11.043483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:11.043835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:11.044686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:11.046373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:11.058576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1315/2638] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:54:11.062611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:11.072948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86142dd-e797-43af-8495-c4584d8bf558 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:11.078617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:11.079286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:11.079574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:11.079851 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:11.080574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:11.082156 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822909fc-e42c-41c5-a5d5-2f5e3e41c69b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:11.092738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1324/2639] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:54:11.096954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:11.111839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a529e-c6ab-4b76-98c4-d565eb21dd06 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:11.117993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:11.119008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:11.119349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:11.120936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] PATH_INFO: `/v3/projects/fc628ba22cfb4a8b9efc0199fc65068f/users/12bd353d748b47debb92d32e726475a7/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:11.135643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc628ba22cfb4a8b9efc0199fc65068f, user_id=12bd353d748b47debb92d32e726475a7, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:11.138291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:11.222312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1316/2640] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:11 2022] PUT /identity/v3/projects/fc628ba22cfb4a8b9efc0199fc65068f/users/12bd353d748b47debb92d32e726475a7/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:11.233340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:11.233636 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:11.234040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:11.262025 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] MFA Rules not processed for user `12bd353d748b47debb92d32e726475a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:11.263283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:11.340080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1325/2641] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:11.353889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:11.356097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:11.428495 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:11.443425 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de556d0-7615-4fd2-93de-9fad0f8ddc6e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:11.446908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebd9b2da29ee4612afb617183145553b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:11.447609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:11.447884 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:11.448160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:11.456662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:11.459004 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:11.468441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1317/2642] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:19.678750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:19.680178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:19.737308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a3a2b-b39d-4401-a666-2742260f44b9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:19.741052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:19.741689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:19.741973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:19.742249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:19.744043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:19.807895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:19.809331 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:19.813408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:19.821073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1326/2643] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:21.642817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=8 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:23.502051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:23.504234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:23.524968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c649d-0efb-44e2-877d-b5a54ff5edce service glance] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:23.530186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:23.531936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:23.532222 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:23.532501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] PATH_INFO: `/v3/users/23b63d44bb304e97a38acc6d0b791c95` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:23.533719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b63d44bb304e97a38acc6d0b791c95)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:23.535827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:23.726714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b63d44bb304e97a38acc6d0b791c95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:23.735599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b63d44bb304e97a38acc6d0b791c95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:23.808463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b63d44bb304e97a38acc6d0b791c95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:23.809066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b63d44bb304e97a38acc6d0b791c95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:23.809688 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-257bea47-f3d4-4b4d-a499-e5116cb441f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b63d44bb304e97a38acc6d0b791c95'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:23.831731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1318/2644] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:23 2022] DELETE /identity/v3/users/23b63d44bb304e97a38acc6d0b791c95 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:24.107593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.120702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7c2dd-b38c-418f-9d0b-ff5186bce776 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.190463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.191589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.191910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.192190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] PATH_INFO: `/v3/projects/29c1b9c9f7024f8494e0a9b58fe35710` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.194197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c1b9c9f7024f8494e0a9b58fe35710)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.196245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.303127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c1b9c9f7024f8494e0a9b58fe35710', 'request_id': 'req-fdc17fbe-3448-4a17-961f-721a4b71d6cf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:24.414220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c1b9c9f7024f8494e0a9b58fe35710', 'request_id': 'req-fdc17fbe-3448-4a17-961f-721a4b71d6cf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:24.421355 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c1b9c9f7024f8494e0a9b58fe35710', 'request_id': 'req-fdc17fbe-3448-4a17-961f-721a4b71d6cf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:24.433194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] The token cache is being invalidate because project 29c1b9c9f7024f8494e0a9b58fe35710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:24.433489 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:24.434158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fdc17fbe-3448-4a17-961f-721a4b71d6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:24.437644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1327/2645] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:24 2022] DELETE /identity/v3/projects/29c1b9c9f7024f8494e0a9b58fe35710 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:24.450289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.450611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.450936 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.469801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:24.471826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:24.536075 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1319/2646] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:24.543849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.544140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.544435 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.563003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:24.568001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:24.572715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1328/2647] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:24.576984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.587668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373c8e0-94f5-4bfb-871b-1c26a08c0d5e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.592315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.592906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.593188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.593467 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.594216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:24.594525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.596149 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.607887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1320/2648] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:54:24.612288 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.621585 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1942b5-d2fa-4600-b4f7-5b1f02fec734 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.626278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.626939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.627218 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.627488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.628164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.630577 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.701238 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1329/2649] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:24 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:24.705433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.715822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78062d-e59f-4bd3-85bc-2d85bb855782 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.771849 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.772514 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.772801 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.773078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.773767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.775712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.782214 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:24.890991 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1321/2650] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:24 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:24.892369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.903242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d20752-8653-423b-96cb-b2627593678d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.908569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.909354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.909516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.909798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.910516 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.912992 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.924421 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1330/2651] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:24.928646 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.939255 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e135d-6cb2-45ec-b4df-45db79cfd35d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.944040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.944698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.944983 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.945258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.945977 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:24.947609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb35596-29ac-4741-b71a-961e78dc4ed7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:24.959115 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1322/2652] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:24.963626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:24.973536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730a36-397c-49f7-8bf4-54c414b5259d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:24.978811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:24.979680 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:24.979965 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:24.980246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] PATH_INFO: `/v3/projects/ce3ae8ed116141658017f823c633c2f2/users/0d76fa94b5ff4721bf490efd0699688e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:24.994995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3ae8ed116141658017f823c633c2f2, user_id=0d76fa94b5ff4721bf490efd0699688e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:25.007383 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:25.072254 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1331/2653] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:24 2022] PUT /identity/v3/projects/ce3ae8ed116141658017f823c633c2f2/users/0d76fa94b5ff4721bf490efd0699688e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:25.077589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf3d76-32b9-4b81-9e16-211d93e5af88 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:25.078003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf3d76-32b9-4b81-9e16-211d93e5af88 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:25.078285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf3d76-32b9-4b81-9e16-211d93e5af88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:25.097234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-acaf3d76-32b9-4b81-9e16-211d93e5af88 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:25.102065 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-acaf3d76-32b9-4b81-9e16-211d93e5af88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:25.160725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1323/2654] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:25.171144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:25.172874 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:25.202580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:25.232261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4985feb-9bc0-4092-88bc-d9be372cf04b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:25.236014 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:25.236655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:25.236936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:25.237209 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:25.247213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:25.248653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098de2e-45f9-42a7-9981-a699bd33e23b service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:25.261050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1332/2655] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:25.480040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:25.480355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:25.480637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:25.499846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] MFA Rules not processed for user `1d04261c643948c0b7d9a20579c52dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:25.505999 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:25.563781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1324/2656] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:31.249668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:31.250791 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:31.251248 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:31.305150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] MFA Rules not processed for user `0d76fa94b5ff4721bf490efd0699688e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:31.306756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:31.422785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1333/2657] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:31.439045 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:31.441636 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:31.551247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:31.567940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2005da-84d1-4354-858e-9dffe31f585a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:31.573749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:31.574412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:31.574733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:31.575020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:31.584671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:31.586342 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3634e-367b-482d-ad64-af84f4bb5ea4 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:31.596848 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1325/2658] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:32.043513 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:32.044913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:32.060081 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decccc03-190e-468e-9cc8-dd27fb056ad3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:32.064512 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:32.065373 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:32.065655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:32.065928 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] PATH_INFO: `/v3/users/12bd353d748b47debb92d32e726475a7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:32.067112 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12bd353d748b47debb92d32e726475a7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:32.069690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:32.276293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12bd353d748b47debb92d32e726475a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:32.284003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12bd353d748b47debb92d32e726475a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:32.284715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12bd353d748b47debb92d32e726475a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:32.290813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12bd353d748b47debb92d32e726475a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:32.374401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12bd353d748b47debb92d32e726475a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:32.400678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1334/2659] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:32 2022] DELETE /identity/v3/users/12bd353d748b47debb92d32e726475a7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:32.404887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:32.405201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:32.405457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:32.424620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:32.425866 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:32.495274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1326/2660] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:32.504565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:32.511020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:32.551058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:32.628763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66a0a6-6c74-43cd-85c9-c6a66b9179c0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:32.633502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:32.634177 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:32.634466 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:32.635454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:32.653210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:32.654840 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fc4c3-0e34-4298-b319-d4e944a96180 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:32.672301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1335/2661] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:33.198722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:33.200268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:33.221941 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273ab75-6fb2-4c7e-85b0-9f1c7620d890 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:33.230429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:33.233428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.233428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.233428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] PATH_INFO: `/v3/projects/fc628ba22cfb4a8b9efc0199fc65068f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.234152 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc628ba22cfb4a8b9efc0199fc65068f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:33.237201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:33.380767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc628ba22cfb4a8b9efc0199fc65068f', 'request_id': 'req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:33.443285 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc628ba22cfb4a8b9efc0199fc65068f', 'request_id': 'req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:33.450232 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc628ba22cfb4a8b9efc0199fc65068f', 'request_id': 'req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:33.462458 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] The token cache is being invalidate because project fc628ba22cfb4a8b9efc0199fc65068f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:33.463028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:33.463879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5edf8a1c-4e6c-4017-8f5e-0e1624a5ea41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:33.466311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1327/2662] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:33 2022] DELETE /identity/v3/projects/fc628ba22cfb4a8b9efc0199fc65068f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:33.471955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.472811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.473250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.499076 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:33.500265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:33.572415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1336/2663] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:33.581611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.582050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.582050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.620042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:33.624609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:33.649306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1328/2664] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:33.650983 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:33.663115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e1e5f-2579-42eb-b0bb-b1b8bb523699 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:33.676294 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:33.677354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.677642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.677922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.679191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:33.679519 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:33.681153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:33.695319 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1337/2665] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 10:54:33.699826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:33.726256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25909488-0b47-496f-ba28-24e3abd6f66b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:33.738996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:33.740090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.740374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.740712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.741357 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:33.743312 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:33.831397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1329/2666] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:33 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:33.832123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:33.847540 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8f91-0ac3-4749-8102-0513b5567cb0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:33.982237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:33.982956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:33.983289 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:33.983521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:33.984219 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:33.986140 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.094819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1338/2667] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:33 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:34.099055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.109333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d6bf-98c9-4938-ba58-62e67c7be367 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.115965 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.116759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.117049 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.117328 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.118051 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.119676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc19105-4304-4a0a-af82-726bea1e2bc0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.131596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1330/2668] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:34.139469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.147745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c35faa-3aba-4e75-93d4-d6aae8031113 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.150789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4e70-ac21-4d56-a735-62334e4f74b8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.151081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4e70-ac21-4d56-a735-62334e4f74b8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.151402 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4e70-ac21-4d56-a735-62334e4f74b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.164169 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.164856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.165143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.165438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.166164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.167898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.176413 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a6ec4e70-ac21-4d56-a735-62334e4f74b8 None None] MFA Rules not processed for user `0d76fa94b5ff4721bf490efd0699688e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:34.178510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a6ec4e70-ac21-4d56-a735-62334e4f74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:34.193790 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1339/2669] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:34.195396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.205125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:34.215456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8680f-8a35-4911-be1b-dc8795db325a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.223262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.225207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.225743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.226273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] PATH_INFO: `/v3/projects/ab48754629074040bbb1fc3e4200738f/users/b7824347a2484ed9b606d5a955a77d84/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.245187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab48754629074040bbb1fc3e4200738f, user_id=b7824347a2484ed9b606d5a955a77d84, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.248202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.258328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:34.262040 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1331/2670] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:34.309070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1340/2671] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:34 2022] PUT /identity/v3/projects/ab48754629074040bbb1fc3e4200738f/users/b7824347a2484ed9b606d5a955a77d84/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:34.315079 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.315658 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.316131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.361652 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] MFA Rules not processed for user `b7824347a2484ed9b606d5a955a77d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:34.363071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:34.473897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1332/2672] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:34.475255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.585536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342d1-a1c5-49a7-92d6-1eedf0b8aaa6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.591843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.593138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.593138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.593442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.594268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.596330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.685760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1341/2673] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:34 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:34.689687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.702018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b819-8dd0-4753-8c5a-e00174cfdf76 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.784474 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.785773 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.786421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.787095 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.788587 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.792406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.879252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1333/2674] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:34 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:54:34.883940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.900788 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac0e76-ef40-459d-ad87-9e808b0db3c7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.906476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.908704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.908704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.908704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.908704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.911005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.933654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1342/2675] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:54:34.934098 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:34.965356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be062dad-fd18-4cc3-b9e3-afc999dcf3da admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:34.972585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:34.975122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:34.975479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:34.975759 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:34.976518 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:34.978472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d41044-bf87-4de5-b1fe-fbef3fd27378 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:34.993976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1334/2676] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:34.995526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.007376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e2af2-b411-4a21-afc9-ba5f3d66974b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.012683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.013583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.013866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.014224 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] PATH_INFO: `/v3/projects/05a96379b2c34967b3cbe923d2527249/users/da749333223d44acbe575a2a557f08e0/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.040980 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a96379b2c34967b3cbe923d2527249, user_id=da749333223d44acbe575a2a557f08e0, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.043957 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.142998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1343/2677] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:34 2022] PUT /identity/v3/projects/05a96379b2c34967b3cbe923d2527249/users/da749333223d44acbe575a2a557f08e0/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:35.152702 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.152985 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.153269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.225217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] MFA Rules not processed for user `da749333223d44acbe575a2a557f08e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:35.228871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:35.281473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:35.331615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1335/2678] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:35.337127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.408590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b00f-041f-4dcf-8829-a6aa9be91728 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.413574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.414244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.414527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.414831 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.415558 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.417455 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.500517 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1344/2679] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:35 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:35.501685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.514695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4677d31-6378-4324-a7dc-509399a9b4ee None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.581424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.582101 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.582428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.582738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.583475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.585742 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.660491 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1336/2680] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:35 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:35.664922 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.679978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a493-e387-4e28-b0e6-011324e4c871 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.685240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.685907 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.686188 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.686467 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.687280 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.689034 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.701941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1345/2681] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:35.706192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.716699 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc67240-a0d2-4a7f-9d87-0815313ce031 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.721433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.722306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.722624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.722904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] PATH_INFO: `/v3/projects/9126a6c508884d739ee72d880da60fe7/users/fa03d12cd3c9421199bf15a2a701b0e7/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.738374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9126a6c508884d739ee72d880da60fe7, user_id=fa03d12cd3c9421199bf15a2a701b0e7, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.742014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.819008 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1337/2682] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:35 2022] PUT /identity/v3/projects/9126a6c508884d739ee72d880da60fe7/users/fa03d12cd3c9421199bf15a2a701b0e7/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:35.823157 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.834210 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e89ce-94f4-4c0d-a1ce-d1103c1e0127 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.846437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:35.892099 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.892767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.893049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.893323 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.894040 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.895924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e56f8-2bc5-4aa3-9c60-1d048a74059a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.905720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1346/2683] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:35.910094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:35.921840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9ca7b-4d26-48e6-baf3-0bf56ed21f83 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:35.926733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:35.927599 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:35.927878 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:35.928150 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] PATH_INFO: `/v3/projects/9126a6c508884d739ee72d880da60fe7/users/fa03d12cd3c9421199bf15a2a701b0e7/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:35.930123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9126a6c508884d739ee72d880da60fe7, user_id=fa03d12cd3c9421199bf15a2a701b0e7, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:35.933025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:35.958918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1338/2684] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:35 2022] PUT /identity/v3/projects/9126a6c508884d739ee72d880da60fe7/users/fa03d12cd3c9421199bf15a2a701b0e7/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:36.012073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:36.012362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:36.012637 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:36.039724 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] MFA Rules not processed for user `fa03d12cd3c9421199bf15a2a701b0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:36.040942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:36.108498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1347/2685] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:36.120598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:36.122362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:36.203581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda219b-caab-4d1d-885e-ecab4070bf8e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:36.207273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:36.208003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:36.208580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:36.208922 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:36.224094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:36.274369 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:36.275717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:36.286619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1339/2686] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:38.375325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:38.386085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ae51-85d0-481e-bcb2-1803b3a51fbc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:38.389710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:38.390510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:38.390807 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:38.391094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:38.452799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:38.454328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:38.464029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1348/2687] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:40.175007 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:40.183841 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f862c2-67fb-48b2-9893-84eae4af213c service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:40.187614 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:40.188264 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:40.188540 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:40.188816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:40.199998 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:40.201446 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:40.213229 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1340/2688] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:42.557683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:42.561422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:42.604752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:42.661021 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42964dc1-291a-4321-9c5c-28c85f4db98d service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:42.667572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:42.668257 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:42.668645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:42.669028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:42.670871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:42.683279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:42.685054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c99f5-1de9-4ed8-bcd1-ddac1795b285 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:42.695614 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1349/2689] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:44.766788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:44.831073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862551fa-14fd-48b4-b39c-95bbd78759ca service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:44.835943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:44.837112 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:44.837391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:44.837663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] PATH_INFO: `/v3/users/b7824347a2484ed9b606d5a955a77d84` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:44.838890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7824347a2484ed9b606d5a955a77d84)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:44.840804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:45.095002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7824347a2484ed9b606d5a955a77d84'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:45.106852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7824347a2484ed9b606d5a955a77d84'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:45.214564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7824347a2484ed9b606d5a955a77d84'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:45.215821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7824347a2484ed9b606d5a955a77d84'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:45.217697 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7824347a2484ed9b606d5a955a77d84'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:45.245131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1341/2690] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:44 2022] DELETE /identity/v3/users/b7824347a2484ed9b606d5a955a77d84 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:54:45.249699 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:45.250183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:45.250279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:45.268071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:45.269270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:45.342620 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1350/2691] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:45.349274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:45.350829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:45.379764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:45.414712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a952-eb7b-4b64-820b-602235a74cd8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:45.418743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:45.419315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:45.419598 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:45.419886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:45.430940 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:45.432579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:45.443326 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1342/2692] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:45.923511 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:45.925007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:45.940899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900b881-f08e-4407-a03e-7272fd2d1820 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:45.947379 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:45.948814 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:45.950485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:45.950485 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] PATH_INFO: `/v3/projects/ab48754629074040bbb1fc3e4200738f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:45.951798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab48754629074040bbb1fc3e4200738f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:45.955471 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:46.083261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab48754629074040bbb1fc3e4200738f', 'request_id': 'req-eb24fa07-cd71-4bb0-9181-e72950a0ca43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.162321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab48754629074040bbb1fc3e4200738f', 'request_id': 'req-eb24fa07-cd71-4bb0-9181-e72950a0ca43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.170315 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab48754629074040bbb1fc3e4200738f', 'request_id': 'req-eb24fa07-cd71-4bb0-9181-e72950a0ca43'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.181921 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] The token cache is being invalidate because project ab48754629074040bbb1fc3e4200738f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:46.182211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.182877 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.184454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1351/2693] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:45 2022] DELETE /identity/v3/projects/ab48754629074040bbb1fc3e4200738f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:46.188498 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:46.198271 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1e4-d534-4131-bf37-5241868622e7 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:46.255227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:46.256103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:46.256384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:46.256725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] PATH_INFO: `/v3/users/da749333223d44acbe575a2a557f08e0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:46.258419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=da749333223d44acbe575a2a557f08e0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:46.260359 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:46.431588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da749333223d44acbe575a2a557f08e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.438934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da749333223d44acbe575a2a557f08e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.526751 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da749333223d44acbe575a2a557f08e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.527390 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da749333223d44acbe575a2a557f08e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.528002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da749333223d44acbe575a2a557f08e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:46.546096 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1343/2694] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:46 2022] DELETE /identity/v3/users/da749333223d44acbe575a2a557f08e0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:47.016001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:47.017514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:47.031212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24fa07-cd71-4bb0-9181-e72950a0ca43 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:47.062494 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:47.089949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:47.090852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:47.091141 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:47.091422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] PATH_INFO: `/v3/projects/05a96379b2c34967b3cbe923d2527249` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:47.092606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a96379b2c34967b3cbe923d2527249)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:47.094506 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:47.213981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a96379b2c34967b3cbe923d2527249', 'request_id': 'req-456323ef-d2a3-4b51-9b68-daeef0f6591e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.304931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a96379b2c34967b3cbe923d2527249', 'request_id': 'req-456323ef-d2a3-4b51-9b68-daeef0f6591e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.311500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a96379b2c34967b3cbe923d2527249', 'request_id': 'req-456323ef-d2a3-4b51-9b68-daeef0f6591e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.322738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] The token cache is being invalidate because project 05a96379b2c34967b3cbe923d2527249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:47.323005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.323645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.326799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1352/2695] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:47 2022] DELETE /identity/v3/projects/05a96379b2c34967b3cbe923d2527249 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:47.329166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:47.330558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:47.344262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab68e-9f37-48fc-a203-b0f1eb2a39fa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:47.386185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:47.400869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:47.401733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:47.402018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:47.402290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] PATH_INFO: `/v3/users/fa03d12cd3c9421199bf15a2a701b0e7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:47.403492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa03d12cd3c9421199bf15a2a701b0e7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:47.405391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:47.621103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa03d12cd3c9421199bf15a2a701b0e7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.627873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa03d12cd3c9421199bf15a2a701b0e7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.693172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa03d12cd3c9421199bf15a2a701b0e7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.693776 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa03d12cd3c9421199bf15a2a701b0e7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.694442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-39fb5863-dfc1-4d18-9060-ba835147caf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa03d12cd3c9421199bf15a2a701b0e7'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:47.713891 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1344/2696] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:47 2022] DELETE /identity/v3/users/fa03d12cd3c9421199bf15a2a701b0e7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:48.083570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.085746 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:48.099270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456323ef-d2a3-4b51-9b68-daeef0f6591e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.167469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.168343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.168939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.168939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] PATH_INFO: `/v3/projects/9126a6c508884d739ee72d880da60fe7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.170537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9126a6c508884d739ee72d880da60fe7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.172434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.284195 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9126a6c508884d739ee72d880da60fe7', 'request_id': 'req-30f55a60-a5b5-4117-951c-07c7e7c64bb4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:48.356245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9126a6c508884d739ee72d880da60fe7', 'request_id': 'req-30f55a60-a5b5-4117-951c-07c7e7c64bb4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:48.362828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9126a6c508884d739ee72d880da60fe7', 'request_id': 'req-30f55a60-a5b5-4117-951c-07c7e7c64bb4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:48.364858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:48.374158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] The token cache is being invalidate because project 9126a6c508884d739ee72d880da60fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:48.374445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:48.375103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30f55a60-a5b5-4117-951c-07c7e7c64bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:48.376644 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1353/2697] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:48 2022] DELETE /identity/v3/projects/9126a6c508884d739ee72d880da60fe7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:48.383407 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.383975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.384094 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.401662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:48.402902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:48.462469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1345/2698] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:48.470283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.470591 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.470876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.488885 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:48.490073 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:48.499437 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1354/2699] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:48.503731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.513711 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a6132-6c9d-4a38-8e13-b293caf541ab None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.518161 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.518835 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.519139 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.519415 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.520132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:54:48.520436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.522002 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.533666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1346/2700] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:54:48.537954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.547594 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b76bcc-88e5-4ef2-af95-70a99ca61096 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.552237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.552868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.553149 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.553420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.554749 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.556735 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.626411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1355/2701] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:54:48 2022] POST /identity/v3/projects => generated 399 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:48.630569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.641856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f93ae-30f2-4648-9b3c-786c7131cc10 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.689712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:48.699725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.700386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.700686 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.701558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.702242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.704163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.824902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1347/2702] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:54:48 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:54:48.829923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.841554 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5979ff1-07b8-48d9-89fb-93a228b08b2c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.846253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.846943 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.847225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.847500 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.848225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.849866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.860655 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1356/2703] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:48.864644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.875829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a4df-2156-44a7-ab0a-99ddf923c86e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.880892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.881584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.882026 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.885449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.885449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.885449 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000ed88-0419-4dc9-ac39-7ef627c564e5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:48.897296 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1348/2704] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:54:48.903964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:48.916448 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce281-b0a3-4a7a-b14e-76bb72956273 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:48.921003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:48.921880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:48.922158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:48.922429 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] PATH_INFO: `/v3/projects/5433a27f4ac94b1f87ea0351bf83e3b7/users/3b2c229c94354914a7f53298cb8cf96e/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:48.936528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5433a27f4ac94b1f87ea0351bf83e3b7, user_id=3b2c229c94354914a7f53298cb8cf96e, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:48.939203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35164b-dddb-4e78-ab0d-5e561e58c3d6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:49.037852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1357/2705] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:54:48 2022] PUT /identity/v3/projects/5433a27f4ac94b1f87ea0351bf83e3b7/users/3b2c229c94354914a7f53298cb8cf96e/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:49.038600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:49.038897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:49.040948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:49.050050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:49.050387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:49.050696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:49.068667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:49.069949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:49.077537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] MFA Rules not processed for user `3b2c229c94354914a7f53298cb8cf96e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:49.078697 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:49.128153 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1349/2706] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:49.146121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1358/2707] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:54:49.158227 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:49.159698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:49.231181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2dc0-d970-4670-a69f-de9ca4c0642f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:49.234993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:49.235650 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:49.235934 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:49.236208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:49.246291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:49.247716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:49.258694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1350/2708] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:49.296122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:49.297564 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:49.378886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9f31a-11b4-4d59-8cf9-c78fd49b6290 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:49.385567 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:49.386222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:49.386510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:49.386838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:49.395682 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:49.397557 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f634c6-411b-4adf-a163-8fa298dd54a9 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:49.407245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1359/2709] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:54:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:54:58.338192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:58.403968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326609e-f141-4c3e-8449-ed5db721f140 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:58.408629 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:58.409496 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:58.409775 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:58.410046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] PATH_INFO: `/v3/users/3b2c229c94354914a7f53298cb8cf96e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:58.411274 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2c229c94354914a7f53298cb8cf96e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:58.413201 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:58.423996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:58.623442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2c229c94354914a7f53298cb8cf96e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:58.635746 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2c229c94354914a7f53298cb8cf96e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:58.709243 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2c229c94354914a7f53298cb8cf96e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:58.709781 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2c229c94354914a7f53298cb8cf96e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:58.710483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2c229c94354914a7f53298cb8cf96e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:58.737872 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1351/2710] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:54:58 2022] DELETE /identity/v3/users/3b2c229c94354914a7f53298cb8cf96e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:54:58.743119 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:58.744193 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:58.744454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:58.758105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:54:58.769738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:54:58.772117 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:58.848066 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1360/2711] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:54:58.859539 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:58.861143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:58.926693 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa223-e4fe-48f7-9b12-05679eef9d79 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:58.930486 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:58.931186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:58.932963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:58.932963 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:58.942251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:58.945177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cffdb-afa2-411b-9d39-fead82e6e15b service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:58.957755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1352/2712] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:54:59.418122 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:54:59.419704 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:54:59.439174 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d61d68-a140-4b99-95db-fb73fd404238 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:54:59.455583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:54:59.460626 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:54:59.461180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:54:59.461581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] PATH_INFO: `/v3/projects/5433a27f4ac94b1f87ea0351bf83e3b7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:54:59.465146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5433a27f4ac94b1f87ea0351bf83e3b7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:54:59.471287 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:54:59.597622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5433a27f4ac94b1f87ea0351bf83e3b7', 'request_id': 'req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:59.687660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5433a27f4ac94b1f87ea0351bf83e3b7', 'request_id': 'req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:59.695164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5433a27f4ac94b1f87ea0351bf83e3b7', 'request_id': 'req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:59.709222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] The token cache is being invalidate because project 5433a27f4ac94b1f87ea0351bf83e3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:54:59.709605 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:59.710401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:54:59.713050 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1361/2713] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:54:59 2022] DELETE /identity/v3/projects/5433a27f4ac94b1f87ea0351bf83e3b7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:06.763659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306230f-a288-43c8-b914-0812534af413 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:06.764113 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306230f-a288-43c8-b914-0812534af413 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:06.764223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306230f-a288-43c8-b914-0812534af413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:06.781143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e306230f-a288-43c8-b914-0812534af413 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:06.782379 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e306230f-a288-43c8-b914-0812534af413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:06.837589 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1353/2714] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 10:55:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:06.849804 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:06.851207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:06.920123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadf33-4ae3-4e8a-908d-1a3b1c16e764 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:06.925001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:06.925642 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:06.925926 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:06.926205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:06.935065 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:06.936764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:06.945049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1362/2715] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 10:55:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 10:55:08.063209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306230f-a288-43c8-b914-0812534af413 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:08.064654 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e306230f-a288-43c8-b914-0812534af413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:08.112750 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:08.129996 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306230f-a288-43c8-b914-0812534af413 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:08.133777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:08.134422 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:08.134743 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:08.135023 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:08.136867 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:08.197906 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:08.199456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:08.208572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1354/2716] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:55:10.173635 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:10.175244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:10.205617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:10.252281 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1832bb2-aa06-4d89-96b6-f33b535e5f37 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:10.257165 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:10.258745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:10.258745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:10.258745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] PATH_INFO: `/v3/users/1d04261c643948c0b7d9a20579c52dae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:10.260012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d04261c643948c0b7d9a20579c52dae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:10.262116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:10.450015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d04261c643948c0b7d9a20579c52dae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:10.456923 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d04261c643948c0b7d9a20579c52dae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:10.457716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d04261c643948c0b7d9a20579c52dae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:10.458430 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d04261c643948c0b7d9a20579c52dae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:10.503887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77867353-0006-4ec0-a2af-996b3c19eed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d04261c643948c0b7d9a20579c52dae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:10.521700 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1363/2717] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:55:10 2022] DELETE /identity/v3/users/1d04261c643948c0b7d9a20579c52dae => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:10.775639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:10.805270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453908b-c3ce-45b7-8c83-4c710a6eed41 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:10.866619 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:10.867492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:10.867777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:10.868054 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] PATH_INFO: `/v3/projects/0f5f72a0f2054b89bb61968a2de9f10a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:10.869485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5f72a0f2054b89bb61968a2de9f10a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:10.872018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:10.997840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5f72a0f2054b89bb61968a2de9f10a', 'request_id': 'req-0f176998-02cb-4d9d-b91e-b1a3428af950'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:11.075611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5f72a0f2054b89bb61968a2de9f10a', 'request_id': 'req-0f176998-02cb-4d9d-b91e-b1a3428af950'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:11.082526 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5f72a0f2054b89bb61968a2de9f10a', 'request_id': 'req-0f176998-02cb-4d9d-b91e-b1a3428af950'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:11.094685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] The token cache is being invalidate because project 0f5f72a0f2054b89bb61968a2de9f10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:55:11.094978 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:11.095620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f176998-02cb-4d9d-b91e-b1a3428af950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:11.097298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1355/2718] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:55:10 2022] DELETE /identity/v3/projects/0f5f72a0f2054b89bb61968a2de9f10a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:11.102343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.102743 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.102934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.123452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:11.124676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:11.183645 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1364/2719] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:11.191947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.192235 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.192512 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.211472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:11.212673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:11.221181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1356/2720] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:11.227103 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.237819 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e810-0318-4e83-ab48-f8d5d3ec9f9c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.242391 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.243072 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.243353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.243627 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.244359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:55:11.244664 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.246411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.257763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1365/2721] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:55:11.261946 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.271276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9d5d6-1bb2-478d-b677-4310c20bb5f0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.275914 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.276557 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.276836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.277110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.278519 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.280470 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.290284 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:11.378005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1357/2722] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:55:11 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:11.379200 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.389035 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bc84-7a9a-4b7f-a9d9-1a5dd44f1075 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.414340 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:11.448196 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.448871 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.449330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.449601 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.450428 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.452579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.562292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1366/2723] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:55:11 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:11.566170 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.577611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad8c5-3b26-4825-a3f8-0a584b37503c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.582247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.582949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.583234 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.583665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.584399 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.586261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.598175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1358/2724] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:55:11.602235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.612387 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbb224-9e1f-4809-a395-a34d06d22723 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.617111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.625452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.625452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.625452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.625452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.625452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aedd81-da5d-467b-96e1-07e05a671f52 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.634328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1367/2725] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:55:11.635689 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.646738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228220-3ddb-4aed-81c6-b4d694e424d9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.651433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.652314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.653189 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.653471 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] PATH_INFO: `/v3/projects/4d55719e2fde40fbae42f57dfbb10aad/users/3b924635a5e84d8bbdc93dd144c0deec/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.668668 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d55719e2fde40fbae42f57dfbb10aad, user_id=3b924635a5e84d8bbdc93dd144c0deec, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.671509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.738755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1359/2726] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:55:11 2022] PUT /identity/v3/projects/4d55719e2fde40fbae42f57dfbb10aad/users/3b924635a5e84d8bbdc93dd144c0deec/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:55:11.749792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.750092 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.750377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.778462 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] MFA Rules not processed for user `3b924635a5e84d8bbdc93dd144c0deec`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:11.779740 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:11.844255 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1368/2727] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:11.855380 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:11.857894 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:11.929888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd790-7dde-444c-8a3e-ba5daa197cb5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:11.932596 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:11.933580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:11.933869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:11.934166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:11.943870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:11.946762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:11.958604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1360/2728] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:55:14.667079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:14.668553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:14.726126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6d882-b1e1-46a0-8f31-cb7ff8ab18d8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:14.729611 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:14.730245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:14.730521 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:14.730829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:14.732587 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:14.779738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:14.795085 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:14.796450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9988a-d4fb-4925-a96a-118ddb882281 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:14.806269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1369/2729] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:55:15.027134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:15.037859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ebc0-161e-47c0-a753-3a71b6ccb842 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:15.042404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:15.043298 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:15.043581 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:15.044033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] PATH_INFO: `/v3/users/3b924635a5e84d8bbdc93dd144c0deec` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:15.045299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b924635a5e84d8bbdc93dd144c0deec)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:15.047210 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:15.302638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b924635a5e84d8bbdc93dd144c0deec'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:15.310767 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b924635a5e84d8bbdc93dd144c0deec'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:15.378203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b924635a5e84d8bbdc93dd144c0deec'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:15.378856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b924635a5e84d8bbdc93dd144c0deec'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:15.379485 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b924635a5e84d8bbdc93dd144c0deec'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:15.397784 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1361/2730] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:55:15 2022] DELETE /identity/v3/users/3b924635a5e84d8bbdc93dd144c0deec => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:55:15.402308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:15.405054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:15.405054 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:15.422143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:15.423369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:15.483148 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1370/2731] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:15.493111 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:15.494609 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:15.530417 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:15.556502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2810-9fe3-4ef8-9fd4-6e3110f37042 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:15.557709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:15.558384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:15.558687 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:15.558962 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:15.569286 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:15.570824 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:15.581276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1362/2732] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:55:16.208096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:16.209652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:16.224336 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608dea-705d-4fd9-a95d-06ea73ddfc47 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:16.229256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:16.229865 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:16.230138 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:16.230410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] PATH_INFO: `/v3/projects/4d55719e2fde40fbae42f57dfbb10aad` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:16.231904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d55719e2fde40fbae42f57dfbb10aad)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:16.234033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:16.358075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d55719e2fde40fbae42f57dfbb10aad', 'request_id': 'req-dce8da2e-36f1-4e56-9835-096e153c9d47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:16.441941 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d55719e2fde40fbae42f57dfbb10aad', 'request_id': 'req-dce8da2e-36f1-4e56-9835-096e153c9d47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:16.451359 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d55719e2fde40fbae42f57dfbb10aad', 'request_id': 'req-dce8da2e-36f1-4e56-9835-096e153c9d47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:16.489825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] The token cache is being invalidate because project 4d55719e2fde40fbae42f57dfbb10aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:55:16.502799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:16.502799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:16.502799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1371/2733] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:55:16 2022] DELETE /identity/v3/projects/4d55719e2fde40fbae42f57dfbb10aad => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:21.979077 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:21.980513 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:21.994239 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f89de-d85d-40da-aa6f-f6623919d9e8 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:22.051278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:22.052093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:22.052374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:22.052647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] PATH_INFO: `/v3/users/e2e3fd195fd14b009ba56950dc542cb1` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:22.053834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e3fd195fd14b009ba56950dc542cb1)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:22.055777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:22.237453 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e3fd195fd14b009ba56950dc542cb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.244174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e3fd195fd14b009ba56950dc542cb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.303954 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e3fd195fd14b009ba56950dc542cb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.304568 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e3fd195fd14b009ba56950dc542cb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.305181 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e3fd195fd14b009ba56950dc542cb1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.326896 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1363/2734] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:55:21 2022] DELETE /identity/v3/users/e2e3fd195fd14b009ba56950dc542cb1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:22.585176 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:22.586925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:22.601028 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8da2e-36f1-4e56-9835-096e153c9d47 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:22.659371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:22.660222 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:22.660498 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:22.660767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] PATH_INFO: `/v3/projects/620ea9c4eaa74dc5858cf93d9fcad9f1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:22.661935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620ea9c4eaa74dc5858cf93d9fcad9f1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:22.663849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:22.674312 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:22.798357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620ea9c4eaa74dc5858cf93d9fcad9f1', 'request_id': 'req-e481b41a-92e2-47ce-8b9b-4e34f8312e22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.869369 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620ea9c4eaa74dc5858cf93d9fcad9f1', 'request_id': 'req-e481b41a-92e2-47ce-8b9b-4e34f8312e22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.876868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620ea9c4eaa74dc5858cf93d9fcad9f1', 'request_id': 'req-e481b41a-92e2-47ce-8b9b-4e34f8312e22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.887760 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] The token cache is being invalidate because project 620ea9c4eaa74dc5858cf93d9fcad9f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:55:22.888115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.888822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:22.890348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1372/2735] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:55:22 2022] DELETE /identity/v3/projects/620ea9c4eaa74dc5858cf93d9fcad9f1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:22.894919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:22.896276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:22.909060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204b48-ce04-405f-a2df-24dce38b9a50 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:22.957437 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:22.966573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:22.967440 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:22.967719 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:22.967989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] PATH_INFO: `/v3/users/22cde1a9790e465faea5c592c26df4a0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:22.969140 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=22cde1a9790e465faea5c592c26df4a0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:22.971070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:23.128269 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22cde1a9790e465faea5c592c26df4a0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.135005 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22cde1a9790e465faea5c592c26df4a0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.205713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22cde1a9790e465faea5c592c26df4a0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.206343 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22cde1a9790e465faea5c592c26df4a0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.206976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-caeeb3dc-3a3f-4d2f-aa90-ecad1b396c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22cde1a9790e465faea5c592c26df4a0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.226038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1364/2736] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:55:22 2022] DELETE /identity/v3/users/22cde1a9790e465faea5c592c26df4a0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:23.465705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:23.467214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:23.480329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481b41a-92e2-47ce-8b9b-4e34f8312e22 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:23.536550 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:23.537405 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:23.537686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:23.537960 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] PATH_INFO: `/v3/projects/7e7cda367f7b44fc80a663a162608909` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:23.539147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7cda367f7b44fc80a663a162608909)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:23.541032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:23.652190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7cda367f7b44fc80a663a162608909', 'request_id': 'req-3f795012-b824-4f35-928f-d9adc01cf5bf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.726660 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7cda367f7b44fc80a663a162608909', 'request_id': 'req-3f795012-b824-4f35-928f-d9adc01cf5bf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.733752 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7cda367f7b44fc80a663a162608909', 'request_id': 'req-3f795012-b824-4f35-928f-d9adc01cf5bf'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.744690 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] The token cache is being invalidate because project 7e7cda367f7b44fc80a663a162608909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:55:23.744973 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.745590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f795012-b824-4f35-928f-d9adc01cf5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:55:23.748142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1373/2737] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:55:23 2022] DELETE /identity/v3/projects/7e7cda367f7b44fc80a663a162608909 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:23.753237 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:23.753553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:23.753839 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:23.771376 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:23.772610 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:23.831464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1365/2738] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:23.839643 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:23.839929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:23.840208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:23.858747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:23.861600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:23.867730 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1374/2739] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:23.872438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:23.882246 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca83a8-df8e-4489-a0c9-4b97708ec5af None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:23.886810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:23.887563 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:23.887840 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:23.896169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:23.896169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:55:23.896169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:23.896169 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:23.902602 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1366/2740] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:55:23.908006 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:23.916948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9f28-f9d8-480f-96b9-491093a22223 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:23.922584 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:23.923242 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:23.923524 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:23.923799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:23.924499 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:23.926436 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.024995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:24.033868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1375/2741] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:55:23 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:24.036692 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:24.047391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492a14-7a4c-456a-b674-0e65a2bd17c0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:24.103334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:24.103988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.104267 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.104541 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.105225 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:24.107829 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.114968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:24.293226 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1367/2742] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:55:24 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:24.297337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:24.307459 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a0187-3db8-4db4-85c0-be88d0a7b73f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:24.312039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:24.312688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.312966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.313237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.313946 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:24.315571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.326098 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1376/2743] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:55:24.330434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:24.341405 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc239-30eb-4c95-b032-a87f30133f8b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:24.346124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:24.346819 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.347099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.347373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.348144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:24.350410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab7164-cc32-4722-a1de-72086f6fe808 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.361674 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1368/2744] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:55:24.366003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:24.376335 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14167b02-8ae4-4455-b540-ddf054beb948 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:24.380954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:24.382640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.382640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.382640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] PATH_INFO: `/v3/projects/c99e5deac07a429c9c771ef41d278a46/users/a0aebc4614c448bdb77fb167d4ae3de1/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.397247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99e5deac07a429c9c771ef41d278a46, user_id=a0aebc4614c448bdb77fb167d4ae3de1, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:24.400075 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.481214 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1377/2745] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:55:24 2022] PUT /identity/v3/projects/c99e5deac07a429c9c771ef41d278a46/users/a0aebc4614c448bdb77fb167d4ae3de1/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:24.486321 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17dd31-6aeb-41ca-bd33-7558ca81a9b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.486615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17dd31-6aeb-41ca-bd33-7558ca81a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.486892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17dd31-6aeb-41ca-bd33-7558ca81a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.504364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7e17dd31-6aeb-41ca-bd33-7558ca81a9b5 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:24.505584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7e17dd31-6aeb-41ca-bd33-7558ca81a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:24.565815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1369/2746] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:24.575893 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:24.577578 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:24.634656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2168c3-1a22-4247-b7d4-eba4ecdda8a2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:24.639036 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:24.639683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:24.639991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:24.640268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:24.649415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:24.650866 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:24.660406 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1378/2747] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:55:29.896943 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:29.897889 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:29.903240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:29.950406 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] MFA Rules not processed for user `a0aebc4614c448bdb77fb167d4ae3de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:29.951620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:30.022479 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1370/2748] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:30.086213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.097720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2569c-5a98-41dc-9003-05ef5e892f68 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.102707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.103374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.103656 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.103934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.104654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.106762 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.203590 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1379/2749] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:55:30 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:30.207757 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.219731 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c39a41-0414-4e02-a2ac-997bb5151286 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.283162 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.283869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.284151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.284426 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.285220 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.287180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.390986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:30.399025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1371/2750] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:55:30 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:55:30.402947 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.413329 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023d20-adf4-42e7-9409-1329baf09caa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.421163 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.421862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.422233 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.422510 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.423425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.428314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.445898 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1380/2751] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:55:30.450630 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.483499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efda50-fe25-43e5-875f-dc30cd89b6ac admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.483499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.483499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.483499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.483846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] PATH_INFO: `/v3/projects/42b7c84ab50d49ce86e099d50e293a55/users/7972f412c7a04ee0bd5dbc3a7dba2ab4/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.493001 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7c84ab50d49ce86e099d50e293a55, user_id=7972f412c7a04ee0bd5dbc3a7dba2ab4, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.495729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.567221 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1372/2752] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:55:30 2022] PUT /identity/v3/projects/42b7c84ab50d49ce86e099d50e293a55/users/7972f412c7a04ee0bd5dbc3a7dba2ab4/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:30.572771 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.582115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cf38d-9166-460b-b3f9-a888cfe8da49 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.653326 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.654128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.654410 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.654741 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.655484 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.658828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1abf35-aa6c-4ffe-a149-705a57cfdad3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.668232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1381/2753] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:55:30.673003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:30.683918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea58d-9ce6-4f12-abdc-c7191d296319 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:30.689721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:30.690643 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:30.690927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:30.691200 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] PATH_INFO: `/v3/projects/42b7c84ab50d49ce86e099d50e293a55/users/7972f412c7a04ee0bd5dbc3a7dba2ab4/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:30.693212 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7c84ab50d49ce86e099d50e293a55, user_id=7972f412c7a04ee0bd5dbc3a7dba2ab4, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:30.695932 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:30.709196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1373/2754] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:55:30 2022] PUT /identity/v3/projects/42b7c84ab50d49ce86e099d50e293a55/users/7972f412c7a04ee0bd5dbc3a7dba2ab4/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:55:36.497444 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b2f1-36fe-46b9-a692-8591ca65c070 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:36.497951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b2f1-36fe-46b9-a692-8591ca65c070 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:36.498051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b2f1-36fe-46b9-a692-8591ca65c070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:36.527325 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e5b2b2f1-36fe-46b9-a692-8591ca65c070 None None] MFA Rules not processed for user `7972f412c7a04ee0bd5dbc3a7dba2ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:36.529652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5b2b2f1-36fe-46b9-a692-8591ca65c070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:36.551197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:36.618528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1382/2755] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:55:36.634927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:55:36.636713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:36.739176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae6ce8-13d1-4b55-9c62-b694b6f70c86 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:55:36.744951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:55:36.745607 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:36.745888 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:36.746733 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:36.808436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:55:36.811052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:55:36.820854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1374/2756] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:55:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:55:46.686079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=20 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:46.816349 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:55:55.110110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:55:55.110747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:55:55.111029 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:55:55.129171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] MFA Rules not processed for user `a0aebc4614c448bdb77fb167d4ae3de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:55:55.130366 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:55:55.137159 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1383/2757] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:56:05.123001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:56:11.719753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:56:11.773611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75023ec-c91c-4c04-a630-fb1e3414f8dd service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:56:11.777230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:56:11.777903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:56:11.778188 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:56:11.778465 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:56:11.788933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:56:11.790355 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:56:11.800809 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1375/2758] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:56:21.795662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=26 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:56:36.486503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:56:36.488548 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:56:36.544828 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a247bf-9978-466f-8742-fe427933eab3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:56:36.548416 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:56:36.549056 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:56:36.549341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:56:36.549619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:56:36.551954 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:56:36.565262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:56:36.567420 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:56:36.578183 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1384/2759] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:56:39.390022 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:56:39.398967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19195b-4adf-4048-a9fe-529eaffb1b02 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:56:39.402509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:56:39.403521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:56:39.403800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:56:39.404073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:56:39.405808 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:56:39.419788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:56:39.421143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:56:39.432638 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1376/2760] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:56:46.573371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=26 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:56:49.428037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:56:50.387542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:56:50.388933 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:56:50.400225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ae32f-c9d9-4376-8fbf-19f66b8aee42 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:56:50.403772 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:56:50.404393 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:56:50.404676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:56:50.404951 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:56:50.406696 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:56:50.416659 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:56:50.418015 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087f82b-9c04-43da-9b52-93aef788fdd9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:56:50.426488 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1385/2761] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:00.422402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:57:28.253762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:57:28.265525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9025b-6648-43df-ac1d-e62b8ba1cdbd service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:57:28.271141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:57:28.271817 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:28.272128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:28.272404 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:28.274193 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:28.286800 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:57:28.288672 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:57:28.301028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1377/2762] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:57:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:33.901902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:33.902460 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:33.902625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:33.920357 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] MFA Rules not processed for user `0d76fa94b5ff4721bf490efd0699688e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:57:33.921561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:33.975948 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1386/2763] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:57:34.279419 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:57:34.289263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b5273-b06f-449b-810b-9f634ce8b7ba service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:57:34.293024 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:57:34.293665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:34.293947 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:34.294249 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:34.296052 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:34.305645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:57:34.307119 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:57:34.315395 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1378/2764] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:57:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:34.728970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:57:34.730356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:34.802249 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a1b4d-4601-4a80-a623-9b569b0689f0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:57:34.807903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:57:34.808586 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:34.808833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:34.809111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:34.810896 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:34.822273 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:57:34.823901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:57:34.834180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1387/2765] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:57:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:43.927374 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:57:43.942020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d635-0cf8-405b-81f5-53c6ef9b88dd service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:57:43.949117 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:57:43.949709 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:43.950038 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:43.950325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:43.952329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:43.965777 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:57:43.967438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c633ff-54f9-47d5-9896-9f9ca298bf9b service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:57:43.978322 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1379/2766] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:44.829486 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=57 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:57:47.372822 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:57:47.385059 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66bb5-5109-4be2-9668-7cb2dbeb9256 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:57:47.390625 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fabbe0e78d45f890b21fcf4b348721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:57:47.400433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:57:47.400433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:57:47.400433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:57:47.400433 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:57:47.406290 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:57:47.407897 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:57:47.419211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1388/2767] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:57:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:57:53.973803 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=18 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:57:57.413918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:58:16.142381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:58:16.142907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:58:16.143014 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:58:16.160252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] MFA Rules not processed for user `a0aebc4614c448bdb77fb167d4ae3de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:58:16.161450 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:58:16.170337 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1380/2768] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:58:16.481089 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:58:16.490464 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a13099-aee8-4339-8d3b-884956617ee8 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:58:16.495676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:58:16.495676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:58:16.495676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:58:16.495676 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:58:16.497051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:58:16.507110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:58:16.508792 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:58:16.517110 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1389/2769] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:58:26.155158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:58:26.513181 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:00.589018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:00.590532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:00.613070 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964b5a0-33fa-44a0-a6bc-e9687e82b9c6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:00.618499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:00.619211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:00.619573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:00.619851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:00.621644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:00.635177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:00.636774 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:00.648547 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1381/2770] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:00.743978 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:00.745542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:00.811793 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0292453-40af-4ac9-847e-ee7870d8df71 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:00.817555 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:00.818419 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:00.818734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:00.819009 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] PATH_INFO: `/v3/users/0d76fa94b5ff4721bf490efd0699688e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:00.820207 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d76fa94b5ff4721bf490efd0699688e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:00.822113 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:01.018244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d76fa94b5ff4721bf490efd0699688e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.025136 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d76fa94b5ff4721bf490efd0699688e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.025850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d76fa94b5ff4721bf490efd0699688e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.026639 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d76fa94b5ff4721bf490efd0699688e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.079042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b55a77a7-24be-42c2-ab0b-a637f42efe49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d76fa94b5ff4721bf490efd0699688e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.097617 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1390/2771] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:00 2022] DELETE /identity/v3/users/0d76fa94b5ff4721bf490efd0699688e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:01.339463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:01.352381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcc100-5355-4a1f-b170-3997e5795455 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:01.414128 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:01.415025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.415310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.415582 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] PATH_INFO: `/v3/projects/ce3ae8ed116141658017f823c633c2f2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.416778 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce3ae8ed116141658017f823c633c2f2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:01.418707 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:01.519037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce3ae8ed116141658017f823c633c2f2', 'request_id': 'req-08ba9cf5-87c5-4ce7-8872-5949d8581e46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.577198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce3ae8ed116141658017f823c633c2f2', 'request_id': 'req-08ba9cf5-87c5-4ce7-8872-5949d8581e46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.584195 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce3ae8ed116141658017f823c633c2f2', 'request_id': 'req-08ba9cf5-87c5-4ce7-8872-5949d8581e46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.595082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] The token cache is being invalidate because project ce3ae8ed116141658017f823c633c2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:01.596292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.596844 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-08ba9cf5-87c5-4ce7-8872-5949d8581e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:01.598397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1382/2772] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:01 2022] DELETE /identity/v3/projects/ce3ae8ed116141658017f823c633c2f2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:01.603569 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.603864 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.604146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.622332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:01.623572 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:01.681938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1391/2773] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:01.690571 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.690862 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.691136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.710047 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:01.711292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:01.719177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1383/2774] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:01.724562 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:01.734186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926a274-eb85-40ba-8684-c5eb823129f8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:01.738887 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:01.739532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.739815 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.740203 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.740945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:01.741246 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:01.742859 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:01.754463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1392/2775] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:01.758698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:01.768738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa7d1b-0dbc-4191-8d4f-c05f4d01f627 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:01.773313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:01.773951 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.774233 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.774626 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.775313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:01.777186 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:01.864386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1384/2776] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:01 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:01.868528 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:01.879425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3ea6c-4111-4439-8a08-a1e25c4e3f4b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:01.883134 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:01.936201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:01.936863 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:01.937147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:01.937422 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:01.938097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:01.940751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:02.028858 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1393/2777] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:01 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:02.033384 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:02.043794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184c65-49e1-4e95-87ad-6b816a519246 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:02.048433 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:02.049114 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:02.049397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:02.049676 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:02.050438 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:02.052071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:02.062164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1385/2778] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:02.066668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:02.076581 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf07e47-74aa-4da9-a3a2-1c460bb77e66 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:02.083002 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:02.083653 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:02.083936 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:02.084213 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:02.084924 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:02.086503 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fdb00-5e82-4d7b-8038-d7a0690006cc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:02.097172 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1394/2779] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:02.101247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:02.112058 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71a13-f492-43e1-9466-872fb9a625c4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:02.125806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:02.125806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:02.125806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:02.125806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] PATH_INFO: `/v3/projects/6ca4b813cb55496c8716065622910999/users/ea6ca284639b48b1a1272e74f8213b88/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:02.133104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca4b813cb55496c8716065622910999, user_id=ea6ca284639b48b1a1272e74f8213b88, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:02.135821 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:02.228377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1386/2780] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:02 2022] PUT /identity/v3/projects/6ca4b813cb55496c8716065622910999/users/ea6ca284639b48b1a1272e74f8213b88/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:02.240083 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:02.240384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:02.240668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:02.266918 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] MFA Rules not processed for user `ea6ca284639b48b1a1272e74f8213b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:02.268692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:02.333395 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1395/2781] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:02.343968 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:02.346851 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:02.353280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:02.404499 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f0d9d-ea1f-4788-b7c6-a223d712a262 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:02.409251 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:02.409892 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:02.410224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:02.410504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:02.418879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:02.420386 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c3453-36d1-44cd-922f-2338054d2b0f service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:02.431109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1387/2782] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:12.001377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:12.063143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36c7f-9453-460c-8900-a6b79e54d87e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:12.068606 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:12.069487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:12.069768 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:12.070043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] PATH_INFO: `/v3/users/ea6ca284639b48b1a1272e74f8213b88` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:12.071206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6ca284639b48b1a1272e74f8213b88)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:12.073120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:12.089386 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:12.195955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6ca284639b48b1a1272e74f8213b88'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:12.203632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6ca284639b48b1a1272e74f8213b88'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:12.204376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6ca284639b48b1a1272e74f8213b88'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:12.205013 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6ca284639b48b1a1272e74f8213b88'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:12.268895 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6ca284639b48b1a1272e74f8213b88'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:12.287194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1396/2783] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:12 2022] DELETE /identity/v3/users/ea6ca284639b48b1a1272e74f8213b88 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:12.291902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:12.292420 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:12.292525 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:12.309224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:12.310411 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:12.369009 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1388/2784] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:12.379468 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:12.380891 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:12.438835 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f07dc-d7d0-471e-b3a5-c4c4c3254ddf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:12.443285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:12.443919 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:12.444274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:12.444553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:12.453543 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:12.455001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a0ee5-4385-480b-a5dc-f2e85d6df617 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:12.465729 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1397/2785] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:12.831430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:12.834690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:12.844904 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15677158-d81f-43aa-8b36-fb174b501850 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:12.850268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:12.851151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:12.851432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:12.851700 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] PATH_INFO: `/v3/projects/6ca4b813cb55496c8716065622910999` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:12.852874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca4b813cb55496c8716065622910999)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:12.854822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:12.942600 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca4b813cb55496c8716065622910999', 'request_id': 'req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:13.004109 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca4b813cb55496c8716065622910999', 'request_id': 'req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:13.009874 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca4b813cb55496c8716065622910999', 'request_id': 'req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:13.021388 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] The token cache is being invalidate because project 6ca4b813cb55496c8716065622910999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:13.021675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:13.022308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59367e9a-db3d-4b40-81bf-31c0e3ad0c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:13.023877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1389/2786] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:12 2022] DELETE /identity/v3/projects/6ca4b813cb55496c8716065622910999 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:59:13.028938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.029408 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.029545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.047908 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:13.049105 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:13.107813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1398/2787] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:13.118490 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.118792 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.119068 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.137939 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:13.139184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:13.147899 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1390/2788] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:13.152211 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.162384 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641df47-7c5e-48fd-bb1c-fe7908972a11 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.167711 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.168362 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.168672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.168970 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.169694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:13.169998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.171618 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.183477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1399/2789] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:13.187905 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.198576 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfc9c-4354-4f52-9971-d443bbf4d322 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.203436 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.204202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.204494 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.204779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.205783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.208592 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.298242 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1391/2790] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/projects => generated 357 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:13.302686 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.312544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f25b-0aa0-4afa-9c39-96ef4b0a51b5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.333958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:13.369127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.369720 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.370012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.370314 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.371033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.372925 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.439668 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1400/2791] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:13.444476 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.449988 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:13.455579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a7c4b-0aef-422e-9ac7-1fce0f26afd0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.460929 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.461584 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.461869 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.462147 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.462895 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.469663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.475925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1392/2792] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:13.479839 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.490318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7457b-2278-44e2-ab98-6831466c5433 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.494976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.495629 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.495910 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.496186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.496899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.498483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70509e78-0c28-4d90-a02d-1d63d56d146c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.510244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1401/2793] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:13.520141 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.529908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c3650-c37b-4f48-875f-e9105968511d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.534424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.535333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.535836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.536276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] PATH_INFO: `/v3/projects/45aca2aa9144444f97a4fcc06561ca41/users/8fce7356378e4a60ba5b6daa75a47989/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.552180 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45aca2aa9144444f97a4fcc06561ca41, user_id=8fce7356378e4a60ba5b6daa75a47989, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.554907 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.633633 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1393/2794] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:13 2022] PUT /identity/v3/projects/45aca2aa9144444f97a4fcc06561ca41/users/8fce7356378e4a60ba5b6daa75a47989/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:13.646041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.646365 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.646769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.673292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] MFA Rules not processed for user `8fce7356378e4a60ba5b6daa75a47989`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:13.675007 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:13.742438 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1402/2795] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:13.746953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.809265 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedc9d-5e66-497e-8cf7-d4e076311ea9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.813806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.814504 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.814822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.815143 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.816845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.818828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:13.892657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1394/2796] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/projects => generated 357 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:13.897170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:13.907775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce10ed4-6716-4c6a-84e6-98cda63869f4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:13.965191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:13.965855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:13.966137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:13.966411 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:13.967167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:13.969223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.028544 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1403/2797] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:13 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:14.032464 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.042834 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0a5e-2c66-4089-9010-b0fa578e0110 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.047482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.048131 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.048412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.048712 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.049427 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.051191 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.061919 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1395/2798] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:14.066302 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.076527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d222-5ae7-4fac-8514-c0dce78fa73b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.082067 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.083016 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.083295 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.083571 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] PATH_INFO: `/v3/projects/45ccdd146376434cb234626d147bddfc/users/e823bedba4154d74be7c35088e4a6b51/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.098353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ccdd146376434cb234626d147bddfc, user_id=e823bedba4154d74be7c35088e4a6b51, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.101033 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.166968 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:14.172030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1404/2799] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:14 2022] PUT /identity/v3/projects/45ccdd146376434cb234626d147bddfc/users/e823bedba4154d74be7c35088e4a6b51/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:59:14.176185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.186292 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b85d8-9f20-4b87-b47f-a9a789114b37 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.243159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.243820 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.244103 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.244377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.245097 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.246723 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92645ada-801b-4beb-b37f-7dbb5b7b48b1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.257010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1396/2800] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:14.261334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.271604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480547-0a54-407e-8688-c31bbe01f453 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.276780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.277654 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.277938 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.278241 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] PATH_INFO: `/v3/projects/45ccdd146376434cb234626d147bddfc/users/e823bedba4154d74be7c35088e4a6b51/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.280232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ccdd146376434cb234626d147bddfc, user_id=e823bedba4154d74be7c35088e4a6b51, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.282886 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.293024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1405/2801] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:14 2022] PUT /identity/v3/projects/45ccdd146376434cb234626d147bddfc/users/e823bedba4154d74be7c35088e4a6b51/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:14.303663 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.304142 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.304261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.329666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] MFA Rules not processed for user `e823bedba4154d74be7c35088e4a6b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:14.331847 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:14.358506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:14.400763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1397/2802] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:14.413194 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.415049 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:14.472345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef04-d2fe-4cd4-b5be-1d4cb58905da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.475949 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.476579 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.476856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.477128 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.485825 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.487348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.496838 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1406/2803] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:14.720764 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:14.730611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5573fc9-c9f6-4fcf-b3d4-47f3c8952cea None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:14.734146 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:14.734806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:14.735089 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:14.735364 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:14.791659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:14.793182 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f99285-4985-4a0f-a747-26b3860857ba service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:14.802412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1398/2804] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:24.492588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:24.798053 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=45 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:33.091689 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:33.151827 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de823ea9-5be3-408e-abdc-27549b1067d9 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:33.156306 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:33.157221 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:33.157456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:33.157728 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] PATH_INFO: `/v3/users/8fce7356378e4a60ba5b6daa75a47989` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:33.158914 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fce7356378e4a60ba5b6daa75a47989)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:33.161463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:33.309483 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fce7356378e4a60ba5b6daa75a47989'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:33.316225 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fce7356378e4a60ba5b6daa75a47989'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:33.317531 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fce7356378e4a60ba5b6daa75a47989'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:33.318321 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fce7356378e4a60ba5b6daa75a47989'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:33.381123 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fce7356378e4a60ba5b6daa75a47989'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:33.399032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1407/2805] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:33 2022] DELETE /identity/v3/users/8fce7356378e4a60ba5b6daa75a47989 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:33.403661 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:33.404160 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:33.404281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:33.421842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:33.423081 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:33.483412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1399/2806] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:33.493324 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:33.494766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:33.551096 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dcc77-cb64-42aa-911d-3e23d6b5f651 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:33.555022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:33.555651 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:33.555931 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:33.556204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:33.565143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:33.566619 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:33.576964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1408/2807] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:34.006917 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:34.008333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:34.022701 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b045aa-9853-4713-9a0e-817c29864698 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:34.027167 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:34.028025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:34.028303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:34.028573 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] PATH_INFO: `/v3/projects/45aca2aa9144444f97a4fcc06561ca41` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:34.029749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45aca2aa9144444f97a4fcc06561ca41)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:34.031694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:34.125779 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45aca2aa9144444f97a4fcc06561ca41', 'request_id': 'req-7b162320-2639-4b9d-9e7d-459e1dd91e45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.205434 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45aca2aa9144444f97a4fcc06561ca41', 'request_id': 'req-7b162320-2639-4b9d-9e7d-459e1dd91e45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.212401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45aca2aa9144444f97a4fcc06561ca41', 'request_id': 'req-7b162320-2639-4b9d-9e7d-459e1dd91e45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.223616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] The token cache is being invalidate because project 45aca2aa9144444f97a4fcc06561ca41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:34.223881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.224507 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.226039 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1400/2808] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:34 2022] DELETE /identity/v3/projects/45aca2aa9144444f97a4fcc06561ca41 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:34.230121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:34.239876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b3be0-b356-4e4d-9e27-e641ea02ff8f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:34.258223 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:34.297137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:34.298008 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:34.298292 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:34.298597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] PATH_INFO: `/v3/users/e823bedba4154d74be7c35088e4a6b51` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:34.299779 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e823bedba4154d74be7c35088e4a6b51)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:34.301913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:34.516104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e823bedba4154d74be7c35088e4a6b51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.522981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e823bedba4154d74be7c35088e4a6b51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.523667 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e823bedba4154d74be7c35088e4a6b51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.524279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e823bedba4154d74be7c35088e4a6b51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.599030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98927b39-345a-4cbd-b205-a59359b922ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e823bedba4154d74be7c35088e4a6b51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:34.617334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1409/2809] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:34 2022] DELETE /identity/v3/users/e823bedba4154d74be7c35088e4a6b51 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:59:35.035258 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.037488 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:35.053356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162320-2639-4b9d-9e7d-459e1dd91e45 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.111316 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.112172 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.112456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.112740 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] PATH_INFO: `/v3/projects/45ccdd146376434cb234626d147bddfc` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.114613 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ccdd146376434cb234626d147bddfc)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.116532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.200713 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:35.217620 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ccdd146376434cb234626d147bddfc', 'request_id': 'req-554459bf-0f0c-47d6-a0a5-471c01cec3ac'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:35.284537 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ccdd146376434cb234626d147bddfc', 'request_id': 'req-554459bf-0f0c-47d6-a0a5-471c01cec3ac'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:35.291130 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ccdd146376434cb234626d147bddfc', 'request_id': 'req-554459bf-0f0c-47d6-a0a5-471c01cec3ac'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:35.302628 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] The token cache is being invalidate because project 45ccdd146376434cb234626d147bddfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:35.302915 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:35.303667 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-554459bf-0f0c-47d6-a0a5-471c01cec3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:35.305176 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1401/2810] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:35 2022] DELETE /identity/v3/projects/45ccdd146376434cb234626d147bddfc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:35.310232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.310727 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.310849 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.328769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:35.330229 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:35.387876 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1410/2811] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:35.396368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.396665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.396976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.416717 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:35.417897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:35.425838 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1402/2812] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:35.430045 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.439553 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8847a-ea7d-42b5-9f12-e0c60df92cb3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.444186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.451505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.460167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1411/2813] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:35.464463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.474678 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2ecdc-b650-409f-bd51-6ce5c90d31c4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.479290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.479923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.480202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.480472 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.481156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.483056 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.565134 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1403/2814] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:35 2022] POST /identity/v3/projects => generated 407 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:35.569913 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.580024 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652f6-1ae8-4b49-a16b-904b32b3b039 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.607063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:35.638709 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.639378 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.639674 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.639939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.640628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.642593 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.760199 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1412/2815] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:35 2022] POST /identity/v3/users => generated 304 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:35.764358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.775679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c5489-ff78-4d7a-b40d-312642ac6644 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.780132 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.780785 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.781104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.781375 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.782086 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.783685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.794647 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1404/2816] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:35.798672 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.808505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3981-bed3-44d1-9514-335dfaa18228 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.813068 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.813714 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.813998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.814274 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.815027 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.816600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa5126-4053-4845-8ccd-facc1ba2d36d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.827058 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1413/2817] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:35.831279 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:35.841679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94295305-8757-41a6-adae-ec90a4fc17d5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:35.846306 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:35.847207 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.847483 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.847752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] PATH_INFO: `/v3/projects/f2fddb77e3d044588b8a74d853112742/users/e0beda0edf4e4ded8d7b35064db1b2f2/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:35.862003 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fddb77e3d044588b8a74d853112742, user_id=e0beda0edf4e4ded8d7b35064db1b2f2, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:35.864639 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:35.975492 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1405/2818] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:35 2022] PUT /identity/v3/projects/f2fddb77e3d044588b8a74d853112742/users/e0beda0edf4e4ded8d7b35064db1b2f2/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:35.986813 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:35.987107 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:35.987390 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.016037 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] MFA Rules not processed for user `e0beda0edf4e4ded8d7b35064db1b2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:36.017236 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:36.079940 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1414/2819] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:36.084786 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.146126 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e765-5d30-4777-9440-dba8bb0d5826 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.150770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.151416 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.151695 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.151966 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.152637 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.154510 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.240918 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1406/2820] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:36 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:36.244950 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.256701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a00c85-c80f-4b12-b4ef-fc39d98a156b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.312942 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.313638 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.313868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.314143 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.314862 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.316747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.417204 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1415/2821] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:36 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:36.421164 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.432534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710adff-b956-4549-98e9-07b2dbb33c92 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.437280 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.438032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.438248 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.438583 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.444072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.444072 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.452066 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1407/2822] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:36.456504 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.466412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6ce3-6d4c-4d8b-ac76-c4720435d9dd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.473398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.473398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.473398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.473398 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] PATH_INFO: `/v3/projects/04e14f2676e749c9950775a53c92071f/users/ca32760fa2364e7a9b38e8c1c926943c/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.487341 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e14f2676e749c9950775a53c92071f, user_id=ca32760fa2364e7a9b38e8c1c926943c, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.490977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.552677 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1416/2823] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:36 2022] PUT /identity/v3/projects/04e14f2676e749c9950775a53c92071f/users/ca32760fa2364e7a9b38e8c1c926943c/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:36.556752 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.566949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73645-265a-4e04-8d7a-881f007385b0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.587804 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:36.624196 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.624861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.625263 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.625532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.626261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.627873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c21d34-b1f6-4398-afe0-090a38ecfac3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.637624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1408/2824] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:36.642161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.648055 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:36.653240 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606caee-e0b2-41fd-a349-87cc46c2a168 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.658164 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.659080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.659358 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.659633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] PATH_INFO: `/v3/projects/04e14f2676e749c9950775a53c92071f/users/ca32760fa2364e7a9b38e8c1c926943c/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.662271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e14f2676e749c9950775a53c92071f, user_id=ca32760fa2364e7a9b38e8c1c926943c, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.664975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.674955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1417/2825] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:36 2022] PUT /identity/v3/projects/04e14f2676e749c9950775a53c92071f/users/ca32760fa2364e7a9b38e8c1c926943c/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 10:59:36.685276 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.685572 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.685859 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.714041 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] MFA Rules not processed for user `ca32760fa2364e7a9b38e8c1c926943c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:36.715461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:36.784551 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1409/2826] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:36.795881 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:36.797648 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:36.855857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c6bbf-7b56-412e-b8e0-15a878c7c48c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:36.859482 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:36.860121 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:36.860402 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:36.860678 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:36.870115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:36.871574 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:36.881150 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1418/2827] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:37.260937 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:37.271579 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2081-088c-4eeb-a699-f35595149e51 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:37.275184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:37.275815 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:37.276092 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:37.276362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:37.333125 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:37.334698 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2efae6-990d-4d52-8aa4-edbf29249a55 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:37.343371 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1410/2828] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:59:46.876799 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:47.058272 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:47.118646 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c54561-66b9-423f-8b51-f2fb6f81a927 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:47.123125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:47.123989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:47.124269 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:47.124545 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] PATH_INFO: `/v3/users/e0beda0edf4e4ded8d7b35064db1b2f2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:47.125708 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0beda0edf4e4ded8d7b35064db1b2f2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:47.127705 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:47.309399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0beda0edf4e4ded8d7b35064db1b2f2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:47.316377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0beda0edf4e4ded8d7b35064db1b2f2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:47.317070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0beda0edf4e4ded8d7b35064db1b2f2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:47.317694 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0beda0edf4e4ded8d7b35064db1b2f2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:47.339463 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=80 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:47.381381 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0beda0edf4e4ded8d7b35064db1b2f2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:47.399580 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1419/2829] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:47 2022] DELETE /identity/v3/users/e0beda0edf4e4ded8d7b35064db1b2f2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:47.404179 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:47.404484 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:47.404768 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:47.422100 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:47.424211 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:47.483565 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1411/2830] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:47.493945 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:47.495415 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:47.552154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbdd16-3abf-4a79-86db-94b9aac787cc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:47.556533 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:47.557178 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:47.557465 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:47.557747 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:47.566750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:47.568171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:47.577903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1420/2831] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:48.000293 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:48.005657 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:48.013995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1195ae7-3c75-4859-89d4-f13c081156a1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:48.018671 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:48.019532 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:48.019825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:48.020104 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] PATH_INFO: `/v3/projects/f2fddb77e3d044588b8a74d853112742` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:48.021305 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fddb77e3d044588b8a74d853112742)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:48.023870 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:48.128368 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fddb77e3d044588b8a74d853112742', 'request_id': 'req-9af73c13-0e26-4507-9a2a-114fa04d42bb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.186064 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fddb77e3d044588b8a74d853112742', 'request_id': 'req-9af73c13-0e26-4507-9a2a-114fa04d42bb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.192090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fddb77e3d044588b8a74d853112742', 'request_id': 'req-9af73c13-0e26-4507-9a2a-114fa04d42bb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.203300 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] The token cache is being invalidate because project f2fddb77e3d044588b8a74d853112742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:48.204788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.204788 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.206205 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1412/2832] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:47 2022] DELETE /identity/v3/projects/f2fddb77e3d044588b8a74d853112742 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:48.211070 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:48.221153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1be231-cd8a-4e05-aa23-e04e40c06acf service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:48.245041 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:48.283692 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:48.284566 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:48.284850 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:48.285256 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] PATH_INFO: `/v3/users/ca32760fa2364e7a9b38e8c1c926943c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:48.286469 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca32760fa2364e7a9b38e8c1c926943c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:48.288546 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:48.474623 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca32760fa2364e7a9b38e8c1c926943c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.481399 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca32760fa2364e7a9b38e8c1c926943c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.482053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca32760fa2364e7a9b38e8c1c926943c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.482707 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca32760fa2364e7a9b38e8c1c926943c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.547026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a62e45dd-742f-4531-8136-d7dee704cbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca32760fa2364e7a9b38e8c1c926943c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:48.564589 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1421/2833] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:48 2022] DELETE /identity/v3/users/ca32760fa2364e7a9b38e8c1c926943c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:48.976487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:48.978088 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:48.991067 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c13-0e26-4507-9a2a-114fa04d42bb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.048949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.049799 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.050082 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.050358 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] PATH_INFO: `/v3/projects/04e14f2676e749c9950775a53c92071f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.051569 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e14f2676e749c9950775a53c92071f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.053501 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.130192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:49.146325 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e14f2676e749c9950775a53c92071f', 'request_id': 'req-e598990e-1051-48ae-9c3c-e4c219ed1db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:49.204324 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e14f2676e749c9950775a53c92071f', 'request_id': 'req-e598990e-1051-48ae-9c3c-e4c219ed1db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:49.211432 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e14f2676e749c9950775a53c92071f', 'request_id': 'req-e598990e-1051-48ae-9c3c-e4c219ed1db4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:49.223608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] The token cache is being invalidate because project 04e14f2676e749c9950775a53c92071f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:49.223897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:49.224528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e598990e-1051-48ae-9c3c-e4c219ed1db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:49.226110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1413/2834] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:48 2022] DELETE /identity/v3/projects/04e14f2676e749c9950775a53c92071f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:49.231038 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.231501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.231600 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.251343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:49.252527 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:49.312449 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1422/2835] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:49.321912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801947-398d-466a-9392-5babc6a511aa None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.322457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801947-398d-466a-9392-5babc6a511aa None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.322457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801947-398d-466a-9392-5babc6a511aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.341553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-16801947-398d-466a-9392-5babc6a511aa None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:49.342794 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-16801947-398d-466a-9392-5babc6a511aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:49.350873 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1414/2836] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:59:49.355197 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.364821 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4891b-7ea3-43eb-874e-f355d9ac0919 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.369994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.370703 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.370985 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.371260 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.371979 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:49.372283 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.373868 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.386022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1423/2837] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:49.390042 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801947-398d-466a-9392-5babc6a511aa None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.399567 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801947-398d-466a-9392-5babc6a511aa None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.404240 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.404875 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.405155 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.405428 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.406120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.408037 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.518760 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1415/2838] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:49 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:59:49.523153 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.533227 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b5af-07dd-45ec-8e95-4dbedaf2a2af admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.559334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:49.590216 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.590892 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.591175 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.591450 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.592118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.594132 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.686031 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1424/2839] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:49 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:49.690948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.702203 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7906b3-2db6-4d92-a310-616604438edb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.707334 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.707992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.708273 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.708546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.709289 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.710927 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.721588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1416/2840] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:49.726596 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.736532 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13997fa-5b98-452a-9cdb-7d989724e070 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.742279 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.742964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.743245 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.743515 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.744291 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.745880 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43fec8-99c0-4abe-b987-6da7414009e5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.755604 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1425/2841] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:49.759762 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:49.769473 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad835-b4d4-4c9b-b634-7eb974b6fc68 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:49.774381 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:49.775270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.775543 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.775810 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] PATH_INFO: `/v3/projects/352bca4af36042d3bebf60c2d7ebc444/users/a54235cb189b4ad8a1af3d0b5db7ff25/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.790823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=352bca4af36042d3bebf60c2d7ebc444, user_id=a54235cb189b4ad8a1af3d0b5db7ff25, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:49.793521 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:49.890729 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1417/2842] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:49 2022] PUT /identity/v3/projects/352bca4af36042d3bebf60c2d7ebc444/users/a54235cb189b4ad8a1af3d0b5db7ff25/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:49.952377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:49.952575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:49.952846 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:49.979633 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] MFA Rules not processed for user `a54235cb189b4ad8a1af3d0b5db7ff25`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:49.980939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:50.044170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1426/2843] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:50.048705 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.109913 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dafcc-0358-487e-b13d-b494f7ebc401 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.114477 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.115151 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.115429 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.115755 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.116753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.118660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.189679 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1418/2844] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 10:59:50.193852 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.206026 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a0145-928f-4aba-9be6-55ea58be0dd8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.263756 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.264491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.264688 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.264964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.265640 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.267583 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.376463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1427/2845] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:50 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:50.380979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.395585 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06c5a-8006-40a8-a3f5-9e5ab7ad870f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.401291 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.401995 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.402308 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.402615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.403383 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.404984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.417802 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1419/2846] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:50.422187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.433001 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fef82-0415-4d74-a60d-2031add7e02a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.437991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.438901 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.439180 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.439456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] PATH_INFO: `/v3/projects/3c20e99c8e24437788ff4f42d74e8f11/users/fb213a6f454b44a199318a41ddd08ff2/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.457856 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c20e99c8e24437788ff4f42d74e8f11, user_id=fb213a6f454b44a199318a41ddd08ff2, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.461599 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.529454 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1428/2847] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:50 2022] PUT /identity/v3/projects/3c20e99c8e24437788ff4f42d74e8f11/users/fb213a6f454b44a199318a41ddd08ff2/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:50.533806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.545430 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364bd4-86dc-4753-bc0d-80250752eae8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.569366 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:50.614333 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.615046 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.615327 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.615642 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.616616 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.619015 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0006cf-b7ff-4511-ab85-ebbd69e5ed74 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.632782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1420/2848] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:50.636833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.642275 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:50.648552 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557dee-51ed-443f-a2bc-13c7e0b6ec87 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.655726 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.656622 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.656904 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.657190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] PATH_INFO: `/v3/projects/3c20e99c8e24437788ff4f42d74e8f11/users/fb213a6f454b44a199318a41ddd08ff2/roles/de1cc43c43db46bfba3665cbe83fd0c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.659264 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c20e99c8e24437788ff4f42d74e8f11, user_id=fb213a6f454b44a199318a41ddd08ff2, role_id=de1cc43c43db46bfba3665cbe83fd0c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.661900 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.673039 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1429/2849] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:50 2022] PUT /identity/v3/projects/3c20e99c8e24437788ff4f42d74e8f11/users/fb213a6f454b44a199318a41ddd08ff2/roles/de1cc43c43db46bfba3665cbe83fd0c6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:50.683681 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.683976 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.684313 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.714124 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] MFA Rules not processed for user `fb213a6f454b44a199318a41ddd08ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:50.715257 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:50.790247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1421/2850] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:50.801423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:50.803170 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:50.873977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76735f41-fcc5-42c7-a4f9-48460fb0aff2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:50.878171 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:50.878841 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:50.879127 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:50.879397 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:50.892641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:50.895111 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd496f9-ac44-4717-b96f-4c5e1f350bce service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:50.906412 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1430/2851] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:51.412062 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:51.475187 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f530-20d7-40d8-bd5c-9f56edc6b256 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:51.479861 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:51.480722 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:51.480997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:51.481268 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] PATH_INFO: `/v3/users/a54235cb189b4ad8a1af3d0b5db7ff25` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:51.482421 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a54235cb189b4ad8a1af3d0b5db7ff25)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:51.484362 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:51.697315 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a54235cb189b4ad8a1af3d0b5db7ff25'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:51.698564 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:51.705032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a54235cb189b4ad8a1af3d0b5db7ff25'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:51.768994 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a54235cb189b4ad8a1af3d0b5db7ff25'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:51.769645 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a54235cb189b4ad8a1af3d0b5db7ff25'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:51.770348 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a54235cb189b4ad8a1af3d0b5db7ff25'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:51.789071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1422/2852] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:51 2022] DELETE /identity/v3/users/a54235cb189b4ad8a1af3d0b5db7ff25 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:51.793750 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:51.794252 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:51.794376 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:51.811956 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:51.815244 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:51.873115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1431/2853] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:51.883791 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:51.885185 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:51.942208 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823914a-5d4c-4867-b5ef-a945b2da1f1d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:51.945845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:51.946482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:51.946790 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:51.947069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:51.956311 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:51.957665 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:51.967608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1423/2854] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 10:59:52.332201 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:52.334104 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:52.347716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf6094-3a5c-4b9f-aab0-36855fd00874 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:52.352663 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:52.353602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:52.353966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:52.354318 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] PATH_INFO: `/v3/projects/352bca4af36042d3bebf60c2d7ebc444` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:52.355684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=352bca4af36042d3bebf60c2d7ebc444)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:52.357452 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:52.484191 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352bca4af36042d3bebf60c2d7ebc444', 'request_id': 'req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.558997 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352bca4af36042d3bebf60c2d7ebc444', 'request_id': 'req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.565632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352bca4af36042d3bebf60c2d7ebc444', 'request_id': 'req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.576570 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] The token cache is being invalidate because project 352bca4af36042d3bebf60c2d7ebc444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:52.576873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.577478 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.579032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1432/2855] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:52 2022] DELETE /identity/v3/projects/352bca4af36042d3bebf60c2d7ebc444 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:52.583398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:52.593215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be005d2-9e20-4c2d-990d-9a7f950adc71 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:52.651278 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:52.652144 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:52.652424 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:52.652694 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] PATH_INFO: `/v3/users/fb213a6f454b44a199318a41ddd08ff2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:52.653886 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb213a6f454b44a199318a41ddd08ff2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:52.655854 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:52.835753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb213a6f454b44a199318a41ddd08ff2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.842615 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb213a6f454b44a199318a41ddd08ff2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.895454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb213a6f454b44a199318a41ddd08ff2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.896069 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb213a6f454b44a199318a41ddd08ff2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.896670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-faa14f71-aa52-4c94-b11c-20a339937ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb213a6f454b44a199318a41ddd08ff2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:52.902993 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:52.915175 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1424/2856] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:52 2022] DELETE /identity/v3/users/fb213a6f454b44a199318a41ddd08ff2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:53.281032 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:53.282447 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:53.296293 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeefc84-1531-49fa-8ba6-f3fc83ab1b1f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:53.308982 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:53.354715 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:53.355501 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.355785 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.356063 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] PATH_INFO: `/v3/projects/3c20e99c8e24437788ff4f42d74e8f11` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.357239 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c20e99c8e24437788ff4f42d74e8f11)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:53.359158 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:53.462628 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c20e99c8e24437788ff4f42d74e8f11', 'request_id': 'req-e4b91400-8acd-423b-a254-56f8ea51c1c2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:53.545734 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c20e99c8e24437788ff4f42d74e8f11', 'request_id': 'req-e4b91400-8acd-423b-a254-56f8ea51c1c2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:53.552301 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c20e99c8e24437788ff4f42d74e8f11', 'request_id': 'req-e4b91400-8acd-423b-a254-56f8ea51c1c2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:53.563125 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] The token cache is being invalidate because project 3c20e99c8e24437788ff4f42d74e8f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:53.563418 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:53.564042 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e4b91400-8acd-423b-a254-56f8ea51c1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:53.565575 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1433/2857] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:53 2022] DELETE /identity/v3/projects/3c20e99c8e24437788ff4f42d74e8f11 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:53.571087 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.571398 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.571660 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.590028 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:53.591245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:53.650441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1425/2858] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:53.659348 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.659632 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.660463 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.679137 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:53.680332 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:53.688102 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1434/2859] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:53.692281 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:53.702673 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff8c4a-d3af-476f-b9eb-09abc0ebf4b0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:53.707509 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:53.708156 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.708442 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.708715 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.710093 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:53.710360 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:53.711990 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:53.723721 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1426/2860] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:53.728597 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:53.737915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e934-6df8-44f8-9ada-21ae50d44fcf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:53.743423 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:53.744071 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.744377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.744658 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.745353 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:53.747345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:53.822300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1435/2861] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:53 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:53.826431 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:53.838219 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942cf2-f9d7-4f87-bcf1-74a5824e6a10 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:53.894588 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:53.895275 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:53.895559 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:53.895836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:53.896530 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:53.898843 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:54.033218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1427/2862] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:53 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:54.037286 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:54.047537 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1af6b4-2413-46a9-9897-20fe4e25a418 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:54.052043 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:54.060630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:54.060630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:54.060630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:54.060630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:54.060630 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:54.066561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1436/2863] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:59:54.071018 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:54.082339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7679f7e-8ae4-4ff5-8abc-e4917c70f0da admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:54.087558 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:54.088245 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:54.095078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:54.095078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:54.095078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:54.095078 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e106be6-fbe3-4106-90f0-3aa089b2215e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:54.101553 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1428/2864] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 10:59:54.105671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:54.115445 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606459e6-4606-4e47-ae50-4faeca40a936 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:54.120755 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:54.123769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:54.123769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:54.123769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] PATH_INFO: `/v3/projects/c4d53fb3080148dcb9300725c757b911/users/35b8343410b54504aa14b049c19c09e1/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:54.137139 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d53fb3080148dcb9300725c757b911, user_id=35b8343410b54504aa14b049c19c09e1, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:54.139937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:54.221903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1437/2865] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 10:59:54 2022] PUT /identity/v3/projects/c4d53fb3080148dcb9300725c757b911/users/35b8343410b54504aa14b049c19c09e1/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:54.233192 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:54.233478 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:54.233753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:54.260770 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] MFA Rules not processed for user `35b8343410b54504aa14b049c19c09e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:54.261957 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:54.303033 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:54.326071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1429/2866] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:54.337278 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:54.339308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:54.384833 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:54.396235 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97255cd-73f9-4d80-abea-51d9cd4f45a6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:54.401120 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:54.401767 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:54.407206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:54.407206 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:54.410899 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:54.412313 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f419-a000-44c9-825b-4b842f093c74 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:54.421710 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1438/2867] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:58.306277 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:58.367871 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2338-726b-446c-9eb4-73d4e998b783 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:58.372574 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:58.373435 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:58.373716 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:58.373989 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] PATH_INFO: `/v3/users/35b8343410b54504aa14b049c19c09e1` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:58.375174 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b8343410b54504aa14b049c19c09e1)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:58.377071 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:58.532890 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b8343410b54504aa14b049c19c09e1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:58.539897 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b8343410b54504aa14b049c19c09e1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:58.608670 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b8343410b54504aa14b049c19c09e1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:58.609288 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b8343410b54504aa14b049c19c09e1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:58.609900 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b8343410b54504aa14b049c19c09e1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:58.627624 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1430/2868] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 10:59:58 2022] DELETE /identity/v3/users/35b8343410b54504aa14b049c19c09e1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:58.635698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:58.635698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:58.635698 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:58.651232 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:58.652442 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:58.711514 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1439/2869] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 10:59:58.722261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:58.723744 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:58.780782 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9a89-81e1-43cd-9c3b-60cd6567bdb4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:58.784361 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:58.784992 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:58.785270 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:58.785544 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:58.794883 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:58.796299 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf6a58-10d8-4bbe-ae02-5e1245da48c1 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:58.805528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1431/2870] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 10:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 10:59:59.228977 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.230431 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:59.242917 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbceb1-9639-4903-b2e6-8a8a734e9c14 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.248343 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.249202 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.249481 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.249759 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] PATH_INFO: `/v3/projects/c4d53fb3080148dcb9300725c757b911` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.250994 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d53fb3080148dcb9300725c757b911)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.252894 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.364012 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d53fb3080148dcb9300725c757b911', 'request_id': 'req-b7819456-c95d-4703-b1b8-399ea1459b22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:59.444356 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d53fb3080148dcb9300725c757b911', 'request_id': 'req-b7819456-c95d-4703-b1b8-399ea1459b22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:59.450911 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d53fb3080148dcb9300725c757b911', 'request_id': 'req-b7819456-c95d-4703-b1b8-399ea1459b22'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:59.461702 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] The token cache is being invalidate because project c4d53fb3080148dcb9300725c757b911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 10:59:59.461989 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:59.462683 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b7819456-c95d-4703-b1b8-399ea1459b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 10:59:59.464371 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1440/2871] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 10:59:59 2022] DELETE /identity/v3/projects/c4d53fb3080148dcb9300725c757b911 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 10:59:59.469216 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.469683 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.469806 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.477310 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 10:59:59.488842 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:59.489967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:59.549329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1432/2872] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:59.557855 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.558145 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.558426 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.577022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 10:59:59.578212 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 10:59:59.586389 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1441/2873] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 10:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 10:59:59.590836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.600396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46e5e5-6be1-4cfe-a4fe-72c9f1125176 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.605561 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.606528 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.606836 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.607135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.607852 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 10:59:59.608158 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.609753 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.621930 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1433/2874] 10.222.0.48 () {58 vars in 1152 bytes} [Thu Jul 21 10:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 10:59:59.626108 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.635374 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf87f04-5c0e-481a-9605-639ab161cea9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.640004 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.640666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.640915 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.641190 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.641873 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.643798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.749775 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1442/2875] 10.222.0.48 () {60 vars in 1152 bytes} [Thu Jul 21 10:59:59 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:59.753662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.764011 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5030c-4395-4dbf-83c8-c682e04da412 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.819441 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.820107 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.820389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.820666 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.821356 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.823410 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.930608 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1434/2876] 10.222.0.48 () {60 vars in 1143 bytes} [Thu Jul 21 10:59:59 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 10:59:59.934848 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.945317 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29561f-1f51-420f-ac7c-ee42e32b35d8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.950126 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.950829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.951144 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.951427 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.952208 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.953857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.965270 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1443/2877] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 10:59:59.969329 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 10:59:59.979823 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac31b4-aea0-4399-9e63-805c97ffb462 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 10:59:59.984456 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 10:59:59.985110 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 10:59:59.985396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 10:59:59.985675 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 10:59:59.986397 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 10:59:59.988055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7cdca-9c9f-4c13-b0dd-ec50e2c7cace admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 10:59:59.999756 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1435/2878] 10.222.0.48 () {58 vars in 1121 bytes} [Thu Jul 21 10:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 11:00:00.004053 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:00.013739 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c1ca-e14f-4a13-bc30-2a49fab68b08 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:00.018265 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:00.019173 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:00.019457 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:00.019737 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] PATH_INFO: `/v3/projects/400e7fad8c43490d8510d77e6c10ce87/users/d2f0ca4d2e5d4157a3e595e0787c0b93/roles/a0903d733c974f51970116b3948f5ff8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:00.035991 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=400e7fad8c43490d8510d77e6c10ce87, user_id=d2f0ca4d2e5d4157a3e595e0787c0b93, role_id=a0903d733c974f51970116b3948f5ff8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:00.038613 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:00.107520 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1444/2879] 10.222.0.48 () {60 vars in 1482 bytes} [Thu Jul 21 11:00:00 2022] PUT /identity/v3/projects/400e7fad8c43490d8510d77e6c10ce87/users/d2f0ca4d2e5d4157a3e595e0787c0b93/roles/a0903d733c974f51970116b3948f5ff8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:00:00.120604 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ac99-0448-4ff0-a88d-58fd210592f1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:00.120921 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ac99-0448-4ff0-a88d-58fd210592f1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:00.121197 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ac99-0448-4ff0-a88d-58fd210592f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:00.147236 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-edd7ac99-0448-4ff0-a88d-58fd210592f1 None None] MFA Rules not processed for user `d2f0ca4d2e5d4157a3e595e0787c0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:00.148461 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-edd7ac99-0448-4ff0-a88d-58fd210592f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:00.209845 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:00.213908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1436/2880] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:00.224565 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:00.226268 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:00.233902 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:00.284538 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce055-59e9-4b22-84e4-4b9078ebebbd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:00.288523 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec420ff94214b3ab6a041395c0a491c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:00.289161 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:00.289440 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:00.289713 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:00.298057 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:00.299505 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:00.309118 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1445/2881] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:00:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:00:02.010727 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ec3e-131e-4ec8-baed-0d6df6e96654 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:02.011266 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ec3e-131e-4ec8-baed-0d6df6e96654 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:02.011377 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ec3e-131e-4ec8-baed-0d6df6e96654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:02.029517 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-20a1ec3e-131e-4ec8-baed-0d6df6e96654 None None] MFA Rules not processed for user `d2f0ca4d2e5d4157a3e595e0787c0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:02.030737 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-20a1ec3e-131e-4ec8-baed-0d6df6e96654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:02.037536 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1437/2882] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:03.827146 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:03.828561 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:03.896981 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa591fbf-5c5e-4ee8-8aec-ee51397905f4 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:03.902005 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:03.902681 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:03.902969 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:03.903253 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:03.904995 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:03.916903 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:03.918477 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ce894-f211-4ea3-86c2-7f1a018355f8 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:03.928502 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1446/2883] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:00:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:00:12.023401 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:13.486202 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:13.486856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:13.486856 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:13.505481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] MFA Rules not processed for user `a0aebc4614c448bdb77fb167d4ae3de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:13.506725 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:13.563314 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1438/2884] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 11:00:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:13.924187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=56 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:20.730079 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb3822-d00f-4514-a3f3-610fa99edad4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:20.730684 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb3822-d00f-4514-a3f3-610fa99edad4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:20.730810 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb3822-d00f-4514-a3f3-610fa99edad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:20.779769 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9bdb3822-d00f-4514-a3f3-610fa99edad4 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:20.781350 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9bdb3822-d00f-4514-a3f3-610fa99edad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:20.846168 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1447/2885] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 11:00:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:21.003482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:21.004902 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:21.096863 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf045088-4e7c-454a-a0d8-57221bee0581 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:21.101826 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:21.102469 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:21.102783 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:21.103060 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:21.111789 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:21.113396 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:21.121405 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1439/2886] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 11:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 11:00:21.943186 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758227-9162-437d-96b7-d33dafc372fd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:21.943685 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758227-9162-437d-96b7-d33dafc372fd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:21.943789 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758227-9162-437d-96b7-d33dafc372fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:21.962823 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa758227-9162-437d-96b7-d33dafc372fd None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:21.964003 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa758227-9162-437d-96b7-d33dafc372fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:21.971934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1448/2887] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 11:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:21.984454 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:21.995073 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632088-83e9-4876-b2dd-cee7fd80d41b service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:22.000290 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:22.000923 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:22.001209 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:22.001487 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:22.009612 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:22.011224 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:22.019825 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1440/2888] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 11:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 11:00:22.602641 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364021ac-9469-418e-a1c0-a865f039acdf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:22.602935 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364021ac-9469-418e-a1c0-a865f039acdf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:22.604811 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364021ac-9469-418e-a1c0-a865f039acdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:22.621361 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-364021ac-9469-418e-a1c0-a865f039acdf None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:22.623187 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-364021ac-9469-418e-a1c0-a865f039acdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:22.630377 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1449/2889] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 21 11:00:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:22.642123 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:22.652798 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dfb4-38ae-4c92-8f01-ece588007107 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:22.658261 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:22.659019 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:22.659302 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:22.659580 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:22.667546 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:22.669948 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:22.677903 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1441/2890] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 11:00:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 11:00:23.274322 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:23.274843 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:23.274963 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:23.293116 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] MFA Rules not processed for user `46b53ca68e1a4445962f4604d1b90d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:23.294878 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:23.301958 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1450/2891] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 21 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:23.314122 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:23.325506 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8cdb9-d01d-4ebb-a8dd-4ace40de2f35 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:23.330816 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b4c6a5adbda4403883661e072aba906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:23.331447 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:23.331726 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:23.331997 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:23.341223 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:23.342822 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda6f53-852d-42c2-9553-19008235f398 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:23.350879 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1442/2892] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 21 11:00:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 11:00:26.109414 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:26.172456 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928e5e6-72d1-405b-8ee9-99c3bd2af006 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:26.176937 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:26.177798 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:26.178080 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:26.178354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] PATH_INFO: `/v3/users/d2f0ca4d2e5d4157a3e595e0787c0b93` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:26.179592 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f0ca4d2e5d4157a3e595e0787c0b93)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:26.181487 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:26.349300 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f0ca4d2e5d4157a3e595e0787c0b93'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:26.355250 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:26.357285 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f0ca4d2e5d4157a3e595e0787c0b93'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:26.357988 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f0ca4d2e5d4157a3e595e0787c0b93'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:26.359220 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f0ca4d2e5d4157a3e595e0787c0b93'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:26.426162 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f0ca4d2e5d4157a3e595e0787c0b93'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:26.444608 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1451/2893] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 11:00:26 2022] DELETE /identity/v3/users/d2f0ca4d2e5d4157a3e595e0787c0b93 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:00:26.449256 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:26.449749 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:26.449877 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:26.471981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] MFA Rules not processed for user `1db1bf9bea374f37b124738f1edc4d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 11:00:26.471981 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:26.500025 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:26.528120 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1443/2894] 10.222.0.48 () {58 vars in 957 bytes} [Thu Jul 21 11:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 11:00:26.538115 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:26.539602 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:26.601337 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15e080-5ece-4c03-a70e-05aca5a49209 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:26.605084 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:26.605745 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:26.606030 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:26.606308 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:26.617262 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:26.618751 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:26.630284 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1452/2895] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:00:26.872339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:00:26.873763 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:00:26.887136 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ca1ba-afe3-410c-8b75-613eb708b8c2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:00:26.891827 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:00:26.892690 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:00:26.892973 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:00:26.893247 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] PATH_INFO: `/v3/projects/400e7fad8c43490d8510d77e6c10ce87` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:00:26.894475 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=400e7fad8c43490d8510d77e6c10ce87)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:00:26.896412 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:00:27.003074 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400e7fad8c43490d8510d77e6c10ce87', 'request_id': 'req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:27.064984 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400e7fad8c43490d8510d77e6c10ce87', 'request_id': 'req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:27.070718 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400e7fad8c43490d8510d77e6c10ce87', 'request_id': 'req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:27.082050 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] The token cache is being invalidate because project 400e7fad8c43490d8510d77e6c10ce87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 11:00:27.082335 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:27.082979 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:00:27.084502 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1444/2896] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 11:00:26 2022] DELETE /identity/v3/projects/400e7fad8c43490d8510d77e6c10ce87 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:00:36.626019 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=35,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:00:37.079020 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:07.219556 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:07.220975 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:07.232541 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e5f61-9d72-46d0-99ab-59084fa5ac43 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:07.282094 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:07.282764 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:07.283052 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:07.283330 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:07.285147 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:07.342717 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:07.344142 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:07.353020 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1453/2897] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:02:08.350373 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:08.351837 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:08.365641 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8f132-57b8-4676-ab8c-0d55c1cb0afc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:08.369032 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:08.369685 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:08.369967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:08.370339 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:08.372184 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:08.383090 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:08.384549 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:08.393303 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1445/2898] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:02:08.615964 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:08.617976 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:08.630853 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da754934-557e-40d1-ad95-fa754cbc84cf service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:08.635077 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:08.635857 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:08.636247 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:08.636588 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:08.638738 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:08.713271 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:08.715328 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:08.727998 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1454/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:02:09.908881 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:09.918016 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770826-21c8-449e-87e0-566c408bb917 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:09.921953 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:09.922605 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:09.922887 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:09.923159 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:09.925644 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:09.939520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:09.941949 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:09.953345 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1446/2900] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:02:18.721816 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=82 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:19.948166 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=12 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:21.681425 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:21.690154 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4039-2f9b-4616-93e3-24485e3eeb2c service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:21.693763 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd2ab88b240473aa0067856ec2254f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:21.694396 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:21.695263 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:21.695666 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:21.697476 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:21.764671 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:21.766097 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:21.776780 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1455/2901] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 11:02:28.709534 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:28.712135 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:28.727391 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c283e-b36f-45f6-baae-5be75906f8b4 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:28.732055 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:28.732925 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:28.733206 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:28.733481 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] PATH_INFO: `/v3/users/a0aebc4614c448bdb77fb167d4ae3de1` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:28.734714 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0aebc4614c448bdb77fb167d4ae3de1)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:28.736659 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:28.899662 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0aebc4614c448bdb77fb167d4ae3de1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:28.906520 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0aebc4614c448bdb77fb167d4ae3de1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:28.954262 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0aebc4614c448bdb77fb167d4ae3de1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:28.954967 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0aebc4614c448bdb77fb167d4ae3de1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:28.955611 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0aebc4614c448bdb77fb167d4ae3de1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:28.976745 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1447/2902] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 11:02:28 2022] DELETE /identity/v3/users/a0aebc4614c448bdb77fb167d4ae3de1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:02:29.228701 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:29.240934 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e0cc-696f-4b61-aba1-e4efab98dca5 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:29.299829 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:29.300687 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:29.300966 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:29.301237 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] PATH_INFO: `/v3/projects/c99e5deac07a429c9c771ef41d278a46` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:29.302404 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99e5deac07a429c9c771ef41d278a46)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:29.304434 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:29.440022 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99e5deac07a429c9c771ef41d278a46', 'request_id': 'req-5468e4a5-85e7-42bc-87aa-7e79012c9e51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.508529 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99e5deac07a429c9c771ef41d278a46', 'request_id': 'req-5468e4a5-85e7-42bc-87aa-7e79012c9e51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.514955 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99e5deac07a429c9c771ef41d278a46', 'request_id': 'req-5468e4a5-85e7-42bc-87aa-7e79012c9e51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.525939 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] The token cache is being invalidate because project c99e5deac07a429c9c771ef41d278a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 11:02:29.526226 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.526890 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.534023 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1456/2903] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 11:02:29 2022] DELETE /identity/v3/projects/c99e5deac07a429c9c771ef41d278a46 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:02:29.534218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:29.534218 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:29.547734 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71234-8c05-4d75-a38b-314b5cf67650 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:29.608297 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:29.609177 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:29.609457 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:29.609738 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] PATH_INFO: `/v3/users/7972f412c7a04ee0bd5dbc3a7dba2ab4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:29.610972 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7972f412c7a04ee0bd5dbc3a7dba2ab4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:29.612908 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:29.790578 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7972f412c7a04ee0bd5dbc3a7dba2ab4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.797217 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7972f412c7a04ee0bd5dbc3a7dba2ab4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.861215 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7972f412c7a04ee0bd5dbc3a7dba2ab4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.861846 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7972f412c7a04ee0bd5dbc3a7dba2ab4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.862460 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7972f412c7a04ee0bd5dbc3a7dba2ab4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:29.871010 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:29.881099 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1448/2904] 10.222.0.48 () {58 vars in 1223 bytes} [Thu Jul 21 11:02:29 2022] DELETE /identity/v3/users/7972f412c7a04ee0bd5dbc3a7dba2ab4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:02:30.139491 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:02:30.140929 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:02:30.154345 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468e4a5-85e7-42bc-87aa-7e79012c9e51 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:02:30.178167 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:30.223334 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1db1bf9bea374f37b124738f1edc4d39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549a3510fe3f48278464436ac5f2a88c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:02:30.224205 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:02:30.224490 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:02:30.224770 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] PATH_INFO: `/v3/projects/42b7c84ab50d49ce86e099d50e293a55` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:02:30.226536 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b7c84ab50d49ce86e099d50e293a55)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:02:30.228652 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:02:30.338526 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b7c84ab50d49ce86e099d50e293a55', 'request_id': 'req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:30.428542 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b7c84ab50d49ce86e099d50e293a55', 'request_id': 'req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:30.435401 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b7c84ab50d49ce86e099d50e293a55', 'request_id': 'req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:30.445766 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] The token cache is being invalidate because project 42b7c84ab50d49ce86e099d50e293a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 11:02:30.446051 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:30.446716 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a52ddeb3-1337-4632-9662-7cf1c81ed2e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 11:02:30.448261 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1457/2905] 10.222.0.48 () {58 vars in 1232 bytes} [Thu Jul 21 11:02:30 2022] DELETE /identity/v3/projects/42b7c84ab50d49ce86e099d50e293a55 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 11:02:40.205418 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:02:40.593354 dv-cinder-849800-2-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 11:05:30.486163 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 11:05:30.488389 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:05:30.638828 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd49ffd-fc22-4d09-afab-1b82503b5f38 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 11:05:30.703252 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4d6a7f8781c4139a4d927dc47cf1a36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984cdf29aa8f4164bd840cd10a9902fe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 11:05:30.703912 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 11:05:30.704198 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 11:05:30.704482 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 11:05:30.706986 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 11:05:30.718933 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 11:05:30.720527 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f12f06-b509-4206-a80b-a024ee75936e service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 11:05:30.731975 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1449/2906] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 21 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 11:05:40.727230 dv-cinder-849800-2-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}